intercoder reliability?
[TOP] Intercoder reliability is the widely used term for the extent to which independent coders evaluate a characteristic of a message or artifact and reach the same conclusion. Although in its generic use as an indication of measurement consistency this term is appropriate and is used here, Tinsley and Weiss (1975, 2000) note that the more specific term for the type of consistency required in content analysis is intercoder (or interrater) agreement. They write that while reliability could be based on correlational (or analysis of variance) indices that assess the degree to which “ratings of different judges are the same when expressed as deviations from their means,” intercoder agreement is needed in content analysis because it measures only “the extent to which the different judges tend to assign exactly the same rating to each object” (Tinsley & Weiss, 2000, p. 98); even when intercoder agreement is used for variables at the interval or ratio levels of measurement, actual agreement
