Rethinking Assessment

How can we design and deliver online assessments that take student and instructor academic needs and pandemic-stress levels into account? In-person options, like paper final exams, often don’t always translate well to the online environment and relying on frequent, short online quizzes and assignments, while they help keep our students on track, can put added pressure on already overloaded schedules.

We asked some U of T faculty members to reflect on their experiences teaching online, in particular as most of us are experiencing online and quarantine fatigue, and share what has worked best for their students’ – and their own – academic success and mental health.

During Fall2020, in one of my C-level courses, I noticed that some of the students were overwhelmed or stressed out by the number of articles that they had to read or the final paper they had to write. In Winter2021, I decided to make some minor changes in the course design to help the students to relieve their stress and succeed in my courses. As well as encouraging the students to work in groups of two or three throughout the semester, one of the main strategies that I used was to give the students the opportunity to choose their final method of assessment. This way the students picked the assessment method to their strength (final presentation vs. final paper). This did not require a full redesign of the course, but I did have to discuss the different aspects of each assessment with the students during our weekly synchronous sessions or one-on-one meetings so that the students could choose one of the two assessment methods based on the schedules and strength of the group members. This strategy worked really well for me and the students as they chose the method that would work best for them. I could see the passion and enthusiasm since the choice of the assessment created more motivation, and we all know that students learn when they are motivated!

As well as this adjustment, I also included some group activities that were set up to be completed during the synchronous sessions in breakout groups. This way, the students would get the opportunity to work in a breakout group and receive instant feedback from their peers and instructor. This strategy worked really well as the students worked and shared their ideas in a stress-free environment without worrying to be assessed individually. Using this strategy, the participation in synchronous sessions increased to a great amount, and the students performed better with the presence of their peers.

Related Resources
Assessment Resources on Quercus

We have developed a writing assignment for our large enrollment first year chemistry classes that requires students to write a 500 word paper on an open-ended chemistry topic in CHMA10 and CHMA11 (enrolment ~2200).  Students learn research skills through the completion of a series of scaffolded learning modules on Quercus. These resources show how to access scholarly articles using U of T Library resources including Scifinder and Web of Science.  The essay is delivered in three phases using the online platform peerScholar. The first phase is a draft submission. In the second phase, students are assigned two of essays written by their peers and they are asked to provide feedback on the draft. We have also developed online Quercus modules to help train students in giving good quality feedback to help support this stage of the assignment. The final phase involves incorporating received feedback and final submission. We have held benchmarking sessions during each offering of the assignment to ensure fair and equitable grading using a rubric. The feedback students provide is also graded by the TAs using a rubric.

Even before the pandemic began, my courses featured exams that were entirely application-based, designed to foster my students’ critical and creative thinking skills rather than simply cultivate their memory. Each of my exam questions is centered around the findings of a scientific paper, and I ask my students to explain the results of the paper using the knowledge they have gained in my course. Because students must use their logic to reason out the answer to my questions, even if they have access to their notes or Google while they are writing my exams, they will not simply be able to look up the answers. Thus, I do not need to lock questions, and, indeed, there are often only 24 multiple-choice questions on my 2-hour exams in order to provide adequate time for students to think.

This past semester, I introduced a new aspect to my online exams, establishing Google Docs that student could use during the exam to discuss their ideas about each question with their peers. It’s been interesting for me to have a window into my students’ thoughts as they tackle my exam questions, and many students have commented to me how such collaboration has helped them gain a better grasp of the course material, either because they have to defend their reasoning to their peers or because they get to have someone other than me explain the material to them in a different way that clicks.

We’d heard from many people that capacity to focus is lessened online, so felt our usual fixed 90-minute midterm test block would not be appropriate. Our solution was to break that 90 minutes into two 45-minute timed components. Students had a 48-hour window in which to complete both sections and were able to plan when to do them, and how much of a break they wanted between, to suit their schedule and test-taking style.

We used Quercus quizzes, and while we know some folks have found restricting navigation back to prior questions useful for supporting academic integrity in online tests, we wanted to avoid this if we could—we know we wouldn’t like it as students ourselves. Splitting our midterm had the benefit of providing an element of restricted navigation, while still allowing students to plan their time in each section based on what they personally found easier or harder.

We also made this test open book which we felt was a realistic way for us to acknowledge the differences while learning online. We didn’t make any changes to how we approached setting our questions for the open book test versus how we used to for in-person (closed-book) tests. Our class grade distribution was solidly in-line with previous in-person iterations.

As instructors got back to the drawing board in delivering courses online, the teaching team of a large (~1100 student) first-year core engineering course re-imagined the learning experience from the student point-of-view – specifically for those students transitioning from high school to university.

We decided to start from the ground up, revisiting one of the first tools that many students learn in primary school: time management. As engineers, we turned that into a discovery and resource-management process where students actively (and creatively!) put-together their timetables – daily, weekly and monthly – and reflect on those experiences holistically. This encourages students to become aware of projects, deadlines, as well as the importance of personal “time away from the computer”. Being an assignment largely assessed on completion and critical thinking, students are asked to reflect on their own work/life balance in this “new reality”, especially in the context of mental health.

Course Coordinator:

APS100: Orientation to Engineering
APS106: Fundamentals of Computer Programming

Avi J Cohen

Having been online for 5 years, here is a proven, scalable writing assignment developed by an economist and a writing specialist, integrated into an 800-student introductory economics course with multilingual students and TAs. We teach students how to write an abstract for a general audience, based on an Economist article. This exercise helps students recognize the structure of a good argument, which serves as scaffolding before making their own arguments in longer writing assignments. Students first submit a draft using peer review software (peerScholar, free for U of T students and instructors), and then assess, and are assessed by, other students. With that feedback, students re-write and submit a final draft graded by TAs using detailed rubrics and moderated marking sessions. Student evaluations rate the peer review process as very valuable, and research shows that giving feedback is actually more valuable in developing writing skills than receiving feedback.

The abstract assignment is scaffolded, fitting in between two other assignments – How to Read Critically, and How to (Re-)Write an Op-Ed. All assignment details and resources (including short, instructional videos), are available for anyone to use in developing similar assignments more appropriate for your discipline’s writing formats.

In the Summer Term, I taught 1st, 2nd, and 3rd year Cell & Systems Biology courses, and am currently team teaching several large undergraduate courses (>1000 students). Assessments are particularly a major concern for our students. In order to decrease stress, we have made practice sample assessments available online that are similar in formatting, content, and length. Our students have been using these to take online technical setup issues out of the equation – they can focus on the actual content and material, and making sure that they understand the material at the right level.

Related Resources
Querus Support Resources: Assessments

The biggest thing that I’m doing differently this year is having a “radical generosity” policy for extensions. Students can get extensions on anything for any reason, without any documentation and without penalty. My decision was based on the fact that my priority is for them to learn the material, but I don’t want to tie the assessment to their ability to turn in something by a particular time. Especially this year.

They don’t have to ask for the weekly quiz or the weekly discussion post – I just leave them open and then they catch up as necessary. For assignments, I have them contact their TAs to ask for a specific amount of time so that they are making a specific plan for themselves. But we accept every request.

The outpouring of gratitude and appreciation from the students has honestly been overwhelming. That policy is a big part of it.

In this course of 1200 students, we normally ask students to write a 5-7 page essay, and we have a final exam. Once we transitioned to online teaching, we decided to alter our assignments to make them less high stakes, and more likely to keep students engaged with lectures and reading throughout the semester. To that end we had assignments every week of the semester, and we asked students to complete 9 of the 11 assignments available. As a universal accommodation in the event of emergency or illness, all students were allowed to skip two assignments, in any two weeks they were unable to get the work done, with no penalty.

The assignments themselves took different formats. Some assignments required students to do nothing more than watch the lecture and do the reading and answer questions based on information that was clearly delivered in the week’s material. Such assignments were designed to teach students how to take notes. The second type of assignments included writing prompts. For example, we might ask students to write a thesis statement for an essay that takes a position on the following question: xxx. Or we might ask them to develop an annotated bibliography on Indigenous – settler reconciliation in Canada, using a range of sources. The third type of assignment was a short essay, usually 3-5 paragraphs, on a very specific topic. All assignments included a detailed grading rubric. Once the assignments were handed in, we published an answer key, against which students could compare their own answers, to make grades more transparent.

The upside of such a course structure was that students really did keep up with the course material, rather than trying to catch up by binge watching all of the lectures in 2 days before a final exam. The downside was that it took a lot of TA grading hours, and the administrative burden of the course was high.

Resources

Assessment Feedback in Quercus Template
Academic Integrity and the Role of the Instructor
Open Book Assessment Options (CTSI recorded webinar)
Guide to Quizzes in Quercus
Online Assessments and Accessibility (excel)
Instructor Guide to Supporting Students in Online Learning Environments
Teaching Online/Remotely – Planning for Next Term: Assessing Learning
Take Home and Online Exams (Academic Success, Student Life)
Engaging Students in an Online Environment (TATP)
Online Assessments and Accessibility (excel)
Guide to Assignments in Quercus