Course Evaluations

At the University of Toronto, online course evaluations are conducted to collect formative data for instructors to improve their teaching, to provide summative data for administrative purposes and for program and curriculum review, and to provide members of the university community, including students, with information about teaching and courses at the University. Using a unique Cascaded Course Evaluation Framework that provides contextualized feedback, including at the institutional, divisional and individual levels, the Course Evaluations team provides support to fully implemented units and continues to work to implement the framework across the University. Team members provide leadership in the area of online course evaluations in the national and international higher education community, as evidenced by wide-ranging consultations this past year with 20 institutions across four countries.

See Appendix E for Course Evaluation metrics and data.

ONGOING IMPLEMENTATION WORK WITH MEDICINE, PHARMACY & DENTISTRY

Extensive consultations and staggered integration periods are required when implementing the framework in new divisions and Faculties. For every division, there is an attention to detail and a dedication to providing customized evaluations that fit their context(s). This past year, the CTSI Course Evaluations team worked with the Faculties of Medicine, Pharmacy and Dentistry to design and finalize course evaluation items (instructor and divisional level), pilot courses, and consult on evaluation settings and assessments that best suit the particular and varied needs of these Faculties and their individual units and instructors.

The Course Evaluations team completed implementation with the Faculty of Kinesiology and Physical Education, a culmination of 4 years of consultation and planning. The Course Evaluation team now supports 19 fully implemented undergraduate and graduate divisions across all 3 campuses.

Kathy Vu
Director, PharmD for Pharmacists Program & Assistant Professor, Leslie Dan Faculty of Pharmacy
“I worked with Gregory Hum and Tara Wells to bring online course evaluations to the PharmD for Pharmacists program in the Winter 2018 term. I appreciated their willingness to engage with work at any stage of the process. Their knowledge and experience was very helpful during discussions with our team to develop the program-specific questions. I look forward to working with both again as we analyze the results and share them within the program.”

VALIDATION STUDY

The Course Evaluations team engaged in a large-scale validation study (see link below) of the institutional items of U of T’s Cascaded Course Evaluation Framework. The validation study focused on the evaluation of 3,568 individual course instructors, teaching 11,919 individual course sections, across two academic years (2015/2016 & 2016/2017) for the four largest undergraduate divisions at U of T (Faculty of Applied Science & Engineering, Faculty of Arts & Science, University of Toronto Mississauga and University of Toronto Scarborough). The sample, which included 277,498 submitted evaluation surveys from 54,108 unique students, represented more than 75% of the course evaluation surveys that were collected by the Course Evaluations team in that time period.

The validation study supports the validity and reliability of the institutional items across raters, items and course sections, and contributed to the redesign and revision of various course evaluation materials, including the forthcoming updated interpretation guide for administrators. This work highlights the value of the evidence-based Course Evaluation Framework at the University of Toronto and the value of integrating student feedback into course and assignment design.

https://uoft.me/validation-study

INTERSECTIONS

The Course Evaluations team’s work touches nearly every Faculty and division at U of T, whether the framework is fully implemented within a unit or still in the consultation and planning stages. The team supports Faculties and individual units through every evaluation period (at the conclusion of each term and session) – from individual support for instructors in selecting items and interpreting results, answering student questions on the process, ensuring course and instructor data matches information at the department level, to sharing response rates and overall system engagement with institution-level administration (e.g., the Course Evaluation Advisory Group).

Key collaborations to highlight:

  • Working with the University of Toronto Business Intelligence (UTBI) unit to add the course evaluation data set to the institutional data warehouse.
  • Working with the UTBI and the Academic & Collaborative Technology (ACT) team to establish, test, document and implement a new data verification procedure that greatly streamlines the process of reviewing institutional and departmental evaluation data.
  • Working with the CTSI and TATP programming teams to design and deliver workshops for both instructors and graduate students on course evaluations and how to incorporate the findings into course and assignment design. CTSI consultations are also provided for instructors on how to interpret the data for the purposes of annual reviews, as well as tenure and promotion processes.