Jinghua Liu

Followers

Following

Public Views

Akif Burak Tosun

University of Pittsburgh

Beat Signer

Vrije Universiteit Brussel

Remo Caponi

University of Cologne

Toshiyuki Masui

Keio University

Vivian Motti

George Mason University

Hamid R. Rabiee

Sharif University of Technology

Lev Manovich

Graduate Center of the City University of New York

Reine-Marie Bérard

Centre National de la Recherche Scientifique / French National Centre for Scientific Research

Somnath Tagore

JNGP

Adnan Awad

The University Of Jordan

Interests

Uploads

Papers by Jinghua Liu

Smoothing and Equating Methods Applied to Different Types of Test Score Distributions and Evaluated with Respect to Multiple Equating Criteria

ETS Research Report Series, 2011

ABSTRACTIn equating research and practice, equating functions that are smooth are typically assum... more ABSTRACTIn equating research and practice, equating functions that are smooth are typically assumed to be more accurate than equating functions with irregularities. This assumption presumes that population test score distributions are relatively smooth. In this study, two examples were used to reconsider common beliefs about smoothing and equating. The first example involves a relatively smooth population test score distribution and the second example involves a population test score distribution with systematic irregularities. Various smoothing and equating methods (presmoothing, equipercentile, kernel, and postsmoothing) were compared across the two examples with respect to how well the test score distributions were reflected in the equating functions, the smoothness of the equating functions, and the standard errors of equating. The smoothing and equating methods performed more similarly in the first example than in the second example. The results of the second example illustrate...

Download

Score Linking Issues Related to Test Content Changes

Statistics for Social and Behavioral Sciences

... The equating transformation should be invariant across subpopulations from the same populatio... more

Multiple Linking in Equating and Random Scale Drift

ETS Research Report Series, 2011

ABSTRACTMaintaining score stability is crucial for an ongoing testing program that administers se... more ABSTRACTMaintaining score stability is crucial for an ongoing testing program that administers several tests per year over many years. One way to stall the drift of the score scale is to use an equating design with multiple links. In this study, we use the operational and experimental SAT® data collected from 44 administrations to investigate the effect of accumulated equating error in equating conversions and the effect of the use of multiple links in equating. No equating error is directly observed or calculated in the study. Instead, we focus on the behavior of the equating conversions after a series of equatings under the nonequivalent groups with anchor test design and analyze the effect of equating error on conversions. It is observed that the single‐link equating conversions drift further away from the operational ones as more equatings are carried out. Analysis of variance is used to decompose the scale score means and the conversion into two major factors: administration mo...

Download

An Application of Score Equity Assessment: Invariance of Linkage of New SAT® to Old SAT Across Gender Groups

Journal of Educational Measurement, 2006

The College Board's SAT® data are used to illustrate how the score equity assessment (SEA) ca... more The College Board's SAT® data are used to illustrate how the score equity assessment (SEA) can help inform the program about equatability. SEA is used to examine whether the content change(s) to the revised new SAT result in differential linking functions across gender groups. Results of population sensitivity analyses are reported on the linkage of the new SAT critical reading (CR) prototype to an old SAT verbal (OV). Based on the criteria used in this study, population invariance was achieved with respect to gender groups.

A Scale Drift Study

ETS Research Report Series, 2009

ABSTRACTThis study examines the stability of the SAT® scale from 1994 to 2001. A 1994 form and a ... more ABSTRACTThis study examines the stability of the SAT® scale from 1994 to 2001. A 1994 form and a 2001 form were readministered in a 2005 SAT administration, and the 1994 form was equated to the 2001 form. The new conversion was compared to the old conversion. Both the verbal and math sections exhibit a similar degree of scale drift, but in opposite directions: the verbal scale has drifted upward, whereas the math scale has drifted downward. We suggest testing programs monitor the score scales periodically by building a testing form schedule that allows a systematic and periodic checking of scale stability.

Download

Test Score Equating Using a Mini-Version Anchor and a Midi Anchor: A Case Study Using SAT® Data

Journal of Educational Measurement, 2011

Assessing the Practical Equivalence of Conversions When Measurement Conditions Change

Journal of Educational Measurement, 2012

At times, the same set of test questions is administered under different measurement conditions t... more At times, the same set of test questions is administered under different measurement conditions that might affect the psychometric properties of the test scores enough to warrant different score conversions for the different conditions. We propose a procedure for assessing the practical equivalence of conversions developed for the same set of test questions but administered under different measurement conditions. This procedure assesses whether the use of separate conversions for each condition has a desirable or undesirable effect. We distinguish effects due to differences in difficulty from effects due to rounding conventions. The proposed procedure provides objective empirical information that assists in deciding to report a common conversion for a set of items or a different conversion for the set of items when the set is administered under different measurement conditions. To illustrate the use of the procedure, we consider the case where a scrambled test form is used along with a base test form. If section order effects are detected between the scrambled and base forms, a decision needs to be made whether to report a single common conversion for both forms or to report separate conversions. Measurement conditions play a very important role in assessment. Kolen and Brennan (2004) listed common measurement characteristics and conditions as one of the necessary requirements of score equating. Inequities in assessment may occur if the need for same measurement conditions is ignored. In the first two editions of Educational Measurement (Lindquist, 1951; Thorndike, 1971) there were chapters by Lorge (1951) and Jones (1971) dedicated to the nature of measurement. Both chapters emphasized the importance of attending to measurement conditions. Jones (1971) noted, In scientific observations, whether direct or indirect, the conditions for observation are carefully specified in terms of time, place, and circumstance.. ..The statement about an observation, necessarily, must contain specification of the conditions under which it is performed. (p. 337) Jones (1971) later added, In general, science demands a reproducibility of observations. Whenever conditions and methods are identical, the observations should be identical (within range of measurement error) unless the object underwent some changes during observation or subsequent to it. The development of apparatus as an aid to measurement has promoted commonality of perceptual judgments among observers and has enhanced opportunities for agreement and reproducibility. (p. 338) These statements remain as relevant today as they were in earlier days of educational measurement; perhaps even more so given the advances in psychometrics and

Download

Smoothing and Equating Methods Applied to Different Types of Test Score Distributions and Evaluated with Respect to Multiple Equating Criteria

ETS Research Report Series, 2011

Download

Score Linking Issues Related to Test Content Changes

Statistics for Social and Behavioral Sciences

... The equating transformation should be invariant across subpopulations from the same populatio... more

Multiple Linking in Equating and Random Scale Drift

ETS Research Report Series, 2011