Assessing Within-Group Agreement: A Critical Examination Of A Random-Group Resampling Approach  

Oleh:

Ludtke, Oliver ; Robitzsch, Alexander

Jenis: Article from Journal - ilmiah internasional
Dalam koleksi: Organizational Research Methods vol. 12 no. 3 (Jul. 2009), page 461-487.
Topik: within-group agreement; intraclass correlation; levels of analysis; multilevel analysi
Fulltext: 461.pdf (413.35KB)

Isi artikel

The measure of within-group agreement most frequently encountered in organizational psychology is the rWG index. The rWG index is determined by comparing the observed group variance among raters with an expected random variance. The most critical issue in calculating the rWG is the choice of an appropriate random distribution that would be expected to follow from raters making their ratings at random. A data-driven approach that uses randomgroup resampling (RGR) procedures to determine the expected random variance has been proposed. In the present study, the application of the RGR procedure will be illustrated with reference to students’ ratings of their mathematics instruction and critically compared with a recently proposed simulation-based approach. It will be shown mathematically that the probability of obtaining statistically significant within-group agreement when applying the RGR procedure strongly depends on the intraclass correlation as well as on the group sizes. Finally, implications for applying the RGR procedure to assess within-group agreement in multilevel data will be discussed.

Opini Anda

Klik untuk menuliskan opini Anda tentang koleksi ini!

Kembali