Rubric Design. Rubrics are usually linked to the sub-sections of the TOC (the assessment points for artifact presentations) however; they may also be linked to main sections. More than one rubric may be applied to a sub-section (usually used only when there are multiple artifacts that shall be assessed separately). We recommend using one comprehensive rubric for each subsection, rather than many, if it all possible. Rubrics generally include: • The name of the Rubric: The short, but descriptive, name of the assessment. • A general description for the whole rubric (optional): This is presented to the assessor on the screen viewed before they start assessment. It may contain specific instructions, the rationale for the use of the rubric, and even research citations that validate the rubric. • The Assessment regime: This indicates the rules and settings that will govern who will do the assessment, and how it happens. • Department: Users may determine if the rubric and its assessment data is to be associated with a specific department. • Criterion 1: This is the first category of observations users will be making using this rubric and will have multiple levels in it representing each of the different scores that are possible with this rubric. Each criterion that makes up the rubric will consist of the following: - A short title - a common description for this criterion (optional, but may contain a specific outline of the required performance at the highest level, or any other information necessary); - The weighting - pre-set to 1.0, but may be changed if needed to raise or lower the scored weighting for this criterion relative to the other criterion in this rubric; - The level of performance labels - e.g. Weakly Expressed, Evident, Mostly Evident or met / not met, and the scores for each label; - Description - this is where objective attributes of performance are stated. We strongly recommend the use of these detailed performance descriptions. They assist assessors significantly because they present clear criteria and help assessors fix their scoring more accurately given what they are actually seeing. Such detailed performance descriptions also clarify for the student what the nature of their performance is, and the expectations for as yet unachieved levels of performance. Such clarity also reduces inter- rater reliability issues with rubrics, and raises the overall validity, reliability, and accuracy of assessment. This pattern is repeated for all remaining criterion in th...
Rubric Design. Initial design We constructed a rubric by a defining a set of criteria, for- mulating a number of levels of accomplishment, and writing verbal descriptors that explain the various levels. We derived the nine criteria from our code quality model [22]. We chose to define four levels of accomplishment, given that we aim for the rubric to be usable in many programming courses. To create a simple progression, we chose the following defini- tions for each: (i) problematic features are present (ii) core quality goals not yet achieved (iii) core quality goals achieved
