Conversion of Proportion-Correct Standard-Setting Judgments to Cutoff Scores on the Item Response Theory y Scale  

Oleh:

Hurtz, Gregory M. ; Jones, J. Patrick ; Jones, Christian N.

Jenis: Article from Journal - ilmiah internasional
Dalam koleksi: Applied Psychological Measurement vol. 32 no. 5 (Jul. 2008), page 385-406.
Topik: standard-setting; Angoff method; item response theory; cutoff scores; model fit; Monte Carlo; simulation
Fulltext: 385.pdf (200.58KB)

Isi artikel

This study compares the efficacy of different strategies for translating item-level, proportioncorrect standard-setting judgments into a y-metric test cutoff score for use with item response theory (IRT) scoring, using Monte Carlo methods. Simulated Angoff-type ratings, consisting of 1,000 independent 75 Item×13 Rater matrices, were generated at five points along the y continuum, at three levels of rater fit to the item characteristics curves, yielding 14,625,000 ratings as the basis of the analyses. These simulated proportion-correct ratings were converted to the IRT y scale using test-level and item-level methods explicated by Kane (1987). Kane’s optimally weighted, item-level conversion method initially produced anomalous results; however, it was discovered that imposing a restriction on the weights avoided these anomalies and rendered the optimally weighted method the most statistically efficient. Six areas for future research are outlined for advancing the integration of these classical standard-setting ratings into IRT methodology.

Opini Anda

Klik untuk menuliskan opini Anda tentang koleksi ini!

Kembali