Setting Standards for Performance Tests: A Pilot Study of a Three-Level Angoff Method  

Oleh:

Yudkowsky, Rachel ; Downing, Steven M. ; Popescu, Mihaela

Jenis: Article from Journal - ilmiah internasional
Dalam koleksi: Academic Medicine (Journal of the Association of American Medical Colleges) vol. 83 no. S10 (Oct. 2008), page S13.

Ketersediaan

Perpustakaan FK Nomor Panggil: A33.K.2008.03 Non-tandon: 1 (dapat dipinjam: 0) Tandon: tidak ada

Lihat Detail Induk

Isi artikel

Background: The Yes/No Angoff method is a standard-setting variant in which judges estimate whether a borderline student would or would not accomplish each item. However, when there is a 50% chance of success, systematically severe or lenient ratings may skew the cut score. This study piloted a three-level Angoff in which items were assigned to "Yes," "No," or "Maybe" categories. Method: Five judges provided three-level Angoff ratings for seven checklists used in a clinical skills exam for fourth-year students. Systematic severity and leniency were simulated by converting all "Maybe" ratings to "Yes" or "No" ratings, respectively. Results: Forty-one percent of 121 items had at least one "Maybe" rating. Ten percent of all ratings were in the "Maybe" category. One judge accounted for 83% of all "Maybe" ratings; three did not use it at all. Case failure rates varied considerably depending on the simulated severity of ratings. Overall failure rates were not substantially impacted. Conclusions: The three-level Angoff retains the cognitive simplicity of the Yes/No Angoff while addressing the challenge of items midrange for the borderline candidate.

Opini Anda

Klik untuk menuliskan opini Anda tentang koleksi ini!

Kembali