Assessment: do we need to broaden our methodological horizons?

Medical Education, 2007

Advances in Health Sciences Education, 2000

As the rapidity with which medical knowledge is generated and disseminated becomes amplified, an increasing emphasis has been placed on the need for physicians to develop the skills necessary for life-long learning. One such skill is the ability to evaluate one's own deficiencies. A ubiquitous finding in the study of self-assessment, however, is that self-ratings are poorly correlated with other performance measures. Still, many educators view the ability to recognize and communicate one's deficiencies as an important component of adult learning. As a result, two studies have been performed in an attempt to improve upon this status quo. First, we tried to re-define the limits within which self-assessments should be used, using Rosenblit and Keil's argument that calibration between perceived and actual performance will be better within taxonomies that are regularly tested (e.g., factual knowledge) compared to those that are not (e.g., conceptual knowledge). Second, we tried to norm reference individuals based on both the performance of their colleagues and their own historical performance on McMaster's Personal Progress Inventory (a multiple choice question test of medical knowledge). While it appears that students are able to (a) make macro-level self-assessments (i.e., to recognize that third year students typically outperform first year students), and (b) judge their performance relatively accurately after the fact, students were unable to predict the percentage of questions they would answer correctly with a testing procedure in which they have had a substantial amount of feedback. Previous test score was a much better predictor of current test performance than were individuals' expectations.

Personnel Psychology, 1988

This study evaluated the validity of self-assessment in a natural setting. It also addressed the possibility of improving the utility of self-assessment by cautioning subjects that their responses could be compared with other existing data on them. Subjects were 357 males, aged 18-19 years, who were being screened for a prestigious military course. During the screening, they participated in small groups for nine days of tests, class exercises, and field simulations supervised by veteran commanders. On the ninth day, ratings were collected from commanders, peer group members, and the subjects themselves on dimensions related to eventual success in the training course. The experimental group was told their reports would be compared with those from other sources; the control group was not. Predictive and convergent validities were examined on three criteria: course success, commander ratings, and peer evaluations. Selfassessments from both experimental and control group were valid however, those of the experimental group did not yield consistently higher validities. Findings are discussed in regard to their practical ramifications. Although self-assessments are often thought to have little veracity, there is reason to argue that individuals are often in the best position to validly assess their own abilities and behaviors and to predict their subsequent conduct. Theories of self-perception (Jones & Nisbett, 1971) have noted that one has access to wide samples of behaviors under varying situations and periods of time. Moreover, one has unique acquaintance with one's own inner states, feelings, and dispositions, as well as with their changes over time. Further, empirical research has demonstrated that individuals are more sensitive than external observers to situational determinants of their behavior and are less likely to overattribute the outcomes of their actions to dispositional factors. Finally, other individuals provide valuable information about actors' behavior. People are continually observing others and comparing themselves with others, as social comparison theories (e.g., Rstinger, 1954) have argued. Levine (1980) summarized the advantages of actors assessing themselves, saying: "we all have a good deal of time to The authors would lie to thank Dr. Michael Hoffman for his valuable contribution. Correspondence and requests for reprints should be addressed to Shaul Fox.

Journal of Clinical Epidemiology, 2003

The term "clinimetrics" was introduced by Feinstein to describe an approach to scale development that ostensibly is different from the more traditional "psychometrics." I argue that, for a number of reasons, it is time for this term to retire from the scene. I show that the clinimetric approach is neither new nor unique, but is rather a subset of psychometrics. Further, because the majority of new developments in scale construction (e.g., new variations of the intraclass correlation, item response theory, structural equation modeling, and cognitive theories) are reported in the psychometric literature, use of the term "clinimetric," especially among people not exposed to traditional test theory, cuts them off from a rich source of information.

1994

Medical Education, 2018

American Psychologist, 2006

Journal of Applied …

Simon C. RILEYc PhD, Senior Lecturer in Obstetrics and Gynaecology (Non-Clinical) Centre for Reproductive Biology, Queen's Medical Research Institute, University of Edinburgh 47 Little France Crescent, Edinburgh EH16 4TJ, Scotland, UK ... E-mail: [email protected]

1991

1993