Assessment with flexible rubric and AI supported marking

In the past, I was responsible for instructing extensive course sections, some even accommodating over 1000 students. Grading assignments within these courses proved to be a significant time investment. On one side, the substantial grading workload had the potential to lead to examiner fatigue. Conversely, maintaining precision in totaling scores and ensuring uniformity in grading across numerous submissions posed challenges for examiners. Of greater significance, however, was the necessity of delivering prompt feedback to students for enhancing their academic progress. The task of furnishing personalized feedback within such extensive courses presented its own set of difficulties.

I use Gradescope to address the above issues to improve marking efficiency and marking quality, as well as to give students feedback promptly. The benefit of Gradescope is particularly strong for large modules.

First, students are required to submit a pdf file of their answers on the Learning Mall. I then upload them in a batch to the Gradescope for marking. With the course roster function, Gradescope is able to match each script to the corresponding student. Also, Gradescope allows students to submit the work themselves.

Here is how Gradescope could help:

• Saving marking time and managing marking load:

Usually, examiners mark all questions of a particular student and then move to the another student. However, Gradescope organises student answers by questions. Examiners mark the answers of one particular question for all students, and then move to another question, which improve marking efficiency. For fill-in-the blank types of questions, Gradescope could recognise the characters and sort answers into different groups. Examiners only need to mark one script in each group, and then every script in the same group will receive the same mark for this question.

Unlike the assignment function on the Learning Mall, which requires the examiners to set up all rubrics in advance, examiners can flexibly add rubrics at any time during the marking process with Gradescope. For example, it is not easy for examiners to list all possible mistakes students could make. Using Gradescope, examiners could add a rubric whenever a new type of mistake is identified during marking.

• Marking consistency:

To be fair to every student, marking consistency is an important factor that examiners need to pay attention to. Marking question by question improves marking consistency. Gradescope enables examiners to conveniently navigate between scripts for the same question.

When multiple markers are involved, another way to ensure marking consistency is to have the same marker making the same question(s). Gradescope allows multiple markers to mark the questions they are responsible for simultaneously.

Moreover, Gradescope keeps the rubrics and corresponding marks awarded or marks deducted in a list for each question. Let’s say, an examiner found the first student made one type of mistake and deducted 2 marks. Then the same type of mistake was found in the script of the 388th student. Without Gradescope, the examiner may have forgotten how many marks have been deducted for this type of mistake. With Gradescope, the examiner added the rubric for this type of mistake when marking the first student, then just chooses the same rubric when marking the 388th student. If after careful consideration, examiners believe the deduction of 2 marks is inappropriate, then changing the deducted marks from 2 to 1 mark for scripts with the same mistake is extremely easy: just change the mark for the specific rubric with one click.

• Free add-up error:

Gradescope will sum marks for each student automatically. There is therefore no adding-up error.

• Releasing individualised feedback:

Gradescope allows the examiners to release the total mark, marking details, rubrics and comments to each student as individualised feedback. Once marks have been released, students have the option to submit regrade requests. Gradescope allows instructors to easily change students’ grades and submit responses to requests.

• Student performance analytics:

Once marking is completed, examiners can get further insight into what students have learned and where they are potentially struggling. Examiners can view statistics for overall performance as well as at the question/rubric item level. Examiners can further tag questions with concepts, learning objectives, book chapters, etc., and then view statistics by tag.

The timely individualised feedback enables students to identify their strengths and weaknesses, and improve their academic performance. Moreover, the detailed feedback enhances students’ confidence in the quality of marking and fairness.

Working in academia often means we need to work on teaching, research and service. Many teachers have difficulties balancing teaching and other work, and also have difficulties balancing teaching workload and teaching quality. The input of teaching, including curriculum design, creation of student activities, assessment design and marking, etc., could potentially have no upper limit in terms of effort and time.

Meanwhile, we have other work that the university expects us to complete. However, if we reduce the input of teaching, the quality might be compromised. We therefore need to take advantage of technology to manage our workloads, complete work efficiently, as well as improve teaching and learning quality. Gradescope is one of these techniques that enables us to save time and enhance marking quality, while improving the students’ learning experience.

• When examiners mark via Gradescope, they may re-enter from SSO and duplicate the marking during the marking process, in order to ensure the marking progress is saved properly.
• Examiners may design answer sheets in a way that students can only write answers in a pre-designed area. For example, answers to question one all appear on the top of the first page, so the examiner could conveniently browse the same area for each script to mark this question. If some students write answers on page 2 instead, then the marker needs a few seconds to find the answers.
• Examiners could use Gradescope for various types of assessments, including coursework and exams.