Kappa Coefficient

Statistical measure of agreement between raters, corrected for chance agreement.

Also known as: κ, kappa statistic, agreement coefficient

The kappa coefficient $κ$ measures how much agreement between two or more raters exceeds what would be expected by chance.

κ = \frac{p _{o} - p _{e}}{1 - p _{e}}

$p_{o}$ is the observed agreement, $p_{e}$ the chance-expected. $κ = 1$ is perfect agreement, $κ = 0$ is chance level. Negative values indicate systematic disagreement.

Used in

MSA Type 5 (attribute)

In the Algorithm Lab

msa/cohen-kappa
msa/fleiss-kappa
msa/weighted-kappa

Sources

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.
AIAG: Measurement Systems Analysis Reference Manual, 4th ed., 2010 — Kap. III-C

Kappa Coefficient

See also

Used in

In the Algorithm Lab

Sources