Graphical Analysis of Test Items

1997

When norm-referenced tests are developed for instructional purposes, to assess the effects of educational programs, or for educational research purposes, it can be very important to conduct item and test analyses. These analyses can evaluate the quality of items and of the test as a whole. Such analyses can also be employed to revise and improve both items and the test as a whole. However, some best practices in item and test analysis are too infrequently used in actual practice. This paper summarizes recommendations for item and test analysis practices as are reported in commonly used textbooks. These practices are determination of item difficulty, item discrimination, and item distractors. Item difficulty is simply the percentage of students taking the test who answered the item correctly. The larger the percentage getting the item right, the easier the item. A good test item discriminates between those who do well on the test and those who do poorly. The item discrimination index and discrimination coefficients can be computed to determine the discriminating power of an item. In addition, analyzing the distractors (incorrect alternatives) is useful in determining the relative usefulness of the decoy items, which should be modified if students consistently fail to select certain multiple choice alternatives. These techniques can help provide empirical information about how tests are performing in real test situations. (Contains 7 tables and 13 references.) (Author/SLD)

Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 2019

The aim of this study is to introduce the jMetric program which is one of the open source programs that can be used in the context of Item Response Theory and Classical Test Theory. In this context, the interface of the program, importing data to the program, a sample analysis, installing the jmetrik and support for the program are discussed. In sample analysis, the answers given by a total of 500 students from state and private schools, to a 10-item math test were analyzed to see whether they shows differentiating item functioning according to the type of school they attend. As a result of the analysis, it was found that two items were showing medium-level Differential Item Functioning (DIF). As a result of the study, it was found that the jMetric program, which is capable of performing Item Response Theory (IRT) analysis for two-category and multi-category items, is open to innovations, especially because it is open-source, and that researchers can easily add the suggested codes to the program and thus the program can be improved. In addition, an advantage of the program is producing visual results related to the analysis through the item characteristic curves.

Item Analysis , 2018

Item analysis (IA) has leaned on statistical tests geared towards evaluating item relevance to different facets, such as test takers, test designers and raters and it informatively evaluates items implementing statistical tests that impinge on construct or psychological attribute measurement. It also warrants minimizing measurement inconsistency. Designing and piloting items that measure what they are intended to measure can be very illuminating to test validation and fairness (Weir, 2005). Hence, items should yield high correlation values or be discarded (Brown, 2005). IA is brought about to decide on whether to retain or weed items out. Alderson, Clapham, and Wall (1995) maintain that the use of IA is beneficial for bias purposes (Bachman, 2004). Not only does IA measure ability but it can also decide on test takers whose futures may be affected because of badly constructed items or unfair judgment. Given the nature of IA, contrivances of multiple sources of information become germane, however disparate they might appear. Three test theories are important in relation to IA: norm-referenced testing (NRT), criterion-referenced testing (CRT) and the Rasch model. Classical IA in NRT is a manner of pre-testing items for difficulty before any official administration (Alderson et al., 1995). First, the item facility (IF) addresses indices of test takers who answer an item correctly. To calculate the IF index, the sum of all correct answers should be divided by the total number of test takers. Second, the item discrimination (ID) can be calculated by identifying upper and lower groups of the total number of test takers: those who gave correct answers and those who did not. The sum of total scores is ordered into a descending order where upper and lower thirds of the group (33% for each group) are determined. To compute the ID, the IFs of the lower group should be deducted from the IFs of the upper group. Finally, distractor analysis concerns analysis of multiple-choice (MC) items and whether distractors can divert test takers' attention from correct answers. It investigates the percentage of upper, middle, and lower subgroups of test takers who answered MC items correctly.

2011

In the standardized and objective evaluation of student performances, the item analysis is a process in which both students' answers and test questions are examined in order to assess the quality and quantity of the items and the test as a whole. All students from some classrooms of primary and middle school were selected to evaluate their performances by testing. On the basis of the analysis results the tests have been re-designed. The results emphasized that item analysis provides valuable information to the teachers to further item modification and future test development and offers educational tools to assist them.

Educational Measurement: Issues and Practice, 2005

Many educational and psychological tests are inherently multidimensional, meaning these tests measure two or more dimensions or constructs. The purpose of this module is to illustrate how test practitioners and researchers can apply multidimensional item response theory (MIRT) to understand better what their tests are measuring, how accurately the different composites of ability are being assessed, and how this information can be cycled back into the test development process. Procedures for conducting MIRT analyses-from obtaining evidence that the test is multidimensional, to modeling the test as multidimensional, to illustrating the properties of multidimensional items graphicallyare described from both a theoretical and a substantive basis. This module also illustrates these procedures using data from a ninth-grade mathematics achievement test. I t concludes with a discussion of future directions in MIRT research.

Educational and Psychological Measurement, 1965

ITEN analysis for the development of new tests and scales is difficult, not because the calculations are complicated, but because the clerical task is immense. A typical scale construction project involves (a) tallying item responses of several hundred subjects to sev-era1 hundred items in high and low criterion groups, (b) obtaining phi coefficients for each item, (c) scoring a provisional scale, (d) tallying the items again for high and low groups on the provisional scale, (e) obtaining phi coefficients, and (f) scoring the revised scale on a cross-validation group. For some purposes additional tallying and scoring is required. To reduce the clerical labor of scale construction, computer programs were mitten to perform the tallying and scoring operations. These programs, which mere written in FORTRAN IV and MAP, are currently set up to run on the IBill 709 or 7090 IBSYS Monitor System. Data Preparation Item response data are punched into cards in any format that is convenient. More than one item may be punched in a single card

TIMSS, 1999

2019

In this essay, General Total Score (GTS)-based (testing) item analysis is discussed in (1) item difficulty analysis; (2) item decomposition; (3) K-dependence coefficient (uncertainty coefficient) and item dependence analysis; (4) Items structure analysis across different populations.

Research Essay, 2019

In this essay, General Total Score (GTS)-based (testing) item analysis is discussed in (1) item difficulty analysis; (2) item decomposition; (3) K-dependence coefficient (uncertainty coefficient) and item dependence analysis.