Agreement and Kappa-Type Indices de Mast, J.January 21st, 2008
FiledJanuary 21st, 2008It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons).
Agreement and Kappa-Type IndicesJanuary 21st, 2008
FiledJanuary 21st, 2008It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons).
Agreement and Kappa-Type IndicesJanuary 21st, 2008
FiledJanuary 21st, 2008Kappa-type indices use the concept of agreement to ex- press the reproducibility of nominal measurements. This arti- cle grounds kappa-type indices in statistical modeling, making explicit the underlying premises and assumptions. We critically review whether the interpretation of the kappa index as a chance- corrected probability of agreement can be substantiated. Further, we show that the so-called paradoxical behavior of the kappa index is explained from the fact that it is a measure of predic- tive association, rather than a pure measure of reproducibility. We discuss a number of alternative forms, critically examining whether they can be translated in tangible real-life interpreta- tions.