Common Contracts
1 similar Research Study contracts
Inter-rater agreement in assigning levels of difficulty to examination questions in Life SciencesResearch Study • September 10th, 2018
Public perception of “declining standards” in school-leaving examinations often accompanies increases in pass rates in school- leaving examinations. “Declining standards” to the public means easier examination papers. The present study evaluates a South African attempt to estimate the level of difficulty, as distinct from cognitive demand, to exit-level examination papers in Life Sciences. A team of four expert raters assigned a level of difficulty ranging from 1 (easy) to 4 (very difficult). Invalid items were assigned a difficulty level of 0. The reference point was “the ideal average South African learner.” Discussion and practice was conducted for 12 examination papers, followed by individual analysis of four examination papers. Inter-rater agreement for the final four papers was low. Raters assigned most items to difficulty levels 1 and 2, indicating that unreliability may be caused by the instrument having too many levels. Raters’ predictions of levels of difficulty supported the
