Estimating Interrater Agreement With The Average Deviation Index : A User's Guide  

Oleh:

Dunlap, William P. ; Burke, Michael J.

Jenis: Article from Journal - ilmiah internasional
Dalam koleksi: Organizational Research Methods vol. 5 no. 2 (2002), page 159-172.
Topik: AGREEMENT; studies; guidelines; standard deviation; measurement; statistical analysis; likert scale; organizational behaviour
Fulltext: 159[1].pdf (122.03KB)

Ketersediaan

Perpustakaan Pusat (Semanggi) Nomor Panggil: OO3.3 Non-tandon: 1 (dapat dipinjam: 0) Tandon: tidak ada

Lihat Detail Induk

Isi artikel

The authors present guidelines for establishing a useful range for interrater agreement and a cutoff for acceptable interrater agreement when using burke, finkelstein and dusig's average deviation (AD) index as well as critical values for tests of statistical significance with the AD index. Under the assumption that judges respond randomly to an item or set of items in a measure, the authors show that acriterion for acceptable interrate agreement or practical significance when using the AD index can be approximated as c / 6, where c is the number of response options for a likert - type item. The resulting values of 0,8 ; 1,2 ; 1,5 and 1,8 are discussed as standards for acceptable itnerrater agreement when using the AD index with 5- , 7-, 9- and 11- point items, respectively. Using similar logic, the AD agreement index and interpretive standard are generalized to the case of a response scale that involves percentages or propositions, rather than discrete categories or at the other extreme, the assessment of interrate agreement with respect to the rating of a single target on a dichotomous item (e. g. yes - no, agree - disagree, true - false item formats). Finally, the usefulness of these guidelines for judging acceptable levels of interrate agreement with respect to the metric (or units) of the original response scale is discussed.

Opini Anda

Klik untuk menuliskan opini Anda tentang koleksi ini!

Kembali