Inter-Annotator AgreementInter-Annotator Agreement • July 29th, 2009
Equation for σˆκˆ also extends to C categories Drawback: κˆ only uses diagonal and marginals of table, discarding most information from the off-diagonal cells
Inter-Annotator AgreementInter-Annotator Agreement • May 13th, 2014
Abstract. This paper discusses different methods of estimating the inter-annotator agreement in manual annotation of Polish coreference and proposes a new BLANC-based annotation agreement metric. The commonly used agreement indicators are calculated for mention detec- tion, semantic head annotation, near-identity markup and coreference resolution.
AN INTER-ANNOTATOR AGREEMENT MEASUREMENT METHODOLOGY FOR THE TURKISH DISCOURSE BANK (TDB)Inter-Annotator Agreement • October 5th, 2010
Siegel (1988, p. 174) explains how to evaluate the value found in Equation 2.3: “Q is distributed approximately as 32 with df = c-1. If the probability associated with the occurrence of when H0 is true of a value as large as the observed value of Q is equal to or less than α, reject H0”.
