Resolution Marking

Resolution marking is when set an Agreement thresholds on the question or rubric area is used to decide agreement between two or more provisional markers for the points they award to a response or Rubric area (mark scheme). If this threshold is met ie the Provisionals agree with each other then the moderating markers don’t need to input anything and marks can be awarded based on a Final marks calculation (average) which is set at the question level.

When Provisionals do not reach an agreement then this is shown in the marking UI on one question or rubric area as a Marker conflict. When the Provisionals agree, the system can act as the moderating marker and aware the average of what the Provisionals have inputted.

• Resolution marking requires double marking to be enabled and at least 2 Provisional markers

• Resolution marking configurations are controlled at the question and rubric level both the agreement threshold and the resolution strategy

• When Provisional markers ‘Disagree’ a human moderator must decide the final marks

• When Provisional markers ‘Agree’ the system can act as the moderating marker must decide the final marks

Creating agreement thresholds

Each question rom any question requiring marking in the question editor go to the marking tab. From here enable resolution marking and select the preferred strategy. The Resolution strategy is how you want the system to decide the final awarded credits based off the provisional input.

Next configure either the overall question or each rubric area to have an agreement threshold. An agreement threshold defines the maximum allowable score difference in points or percentage.

For agreement threshold, a point difference of 1 this means that an area or question is considered in agreement as long as the provisional markers awarded markers are all within 1 point difference. When there are more than two provisional markers, if the difference between any two markers points exceeds the agreement threshold then the marks are considered in disagreement.

When marks are in agreement, then the resolution strategy is used to calculate the final marks for the response.

As an example, if the agreement threshold is set to 1 point, and the resolution strategy is set to average on a rubric area; if one provisional gave 3 marks the other gave 4, this would trigger an agreement to let the system award 3.5.

Configure resolution marking on an exam

From the exam marking page make sure resolution marking is enabled, if you have this enabled but no questions in an attempt have it configured then nothing will change. You can use the exams questions page to check any questions being used in an exam have this enabled. You still need to have human moderating markers in the list incase of disagreements.

Conflicts from disagreements

With resolution marking configured conflicts will show to moderators, backreaders, auditors or anyone with permission to view marks of that attempt. When score conflicts arrived they're highlighted on the marking page by either question or rubric area (depending on how they've been configured). Its the moderators job to go in and resolve these conflicts by deciding the final score.

Moderators can also choose to accept the marks for each question of provisionals, which will copy all feedback, and credits over to their own marking inputs