At the University of Toronto, online course evaluations are conducted to collect formative data for instructors to improve their teaching, to provide summative data for program, curriculum and course review, and to provide members of the university community, including students, with information about their learning experiences at the University. Through the ongoing Course Evaluation Framework implementation, the CTSI Course Evaluation team provides guidance and facilitation for item design, technical and data support, and guidance around the educational uses of course evaluation data. As of this year, the framework is in place in 13 divisions across the University. For course evaluation metrics see Appendix A.
COURSE EVALUATIONS RESOURCES
The Course Evaluations team continues to develop valuable resources for a range of audiences, including academic administrators, staff, instructors and students. These resources support the work of course evaluations at the University in multiple ways including ensuring the smooth technical operation of the course evaluation system, as well as guiding the valid interpretation and use of course evaluations.
This year, CTSI published the University of Toronto Course Evaluation Interpretation Guidelines for Academic Administrators, which aims to enable consistency and increase transparency in how evaluations are interpreted across the institution, aligning with teaching evaluation policies at U of T. This guide gives practical recommendations for critical evaluation processes based on analyses outlined in CTSI’s 2018 report, University of Toronto’s Cascaded Course Evaluation Framework: Validation Study of the Institutional Composite Mean (ICM).
Following the successful launch of Quercus, course evaluation functions, messaging and content were re-integrated into U of T’s new academic toolbox. To increase visibility and usability, a course evaluations button and page were created within Quercus, providing easier access to course evaluations data and prioritizing user engagement with the system.
2018 Course Evaluation Institute (CEI)
- 48 participants at the Institute
- 21 institutions
- 6 international universities
- 10 sessions over 2 days (9 facilitated by U of T)
- 85% of respondents found every session to be a valuable learning experience
- 100% of respondents would attend again and recommend a future CEI to others
The work of the Course Evaluations team spans the scope of nearly every Faculty and division at U of T, from facilitating implementation of the Framework to ongoing operational support through every evaluation period. The Course Evaluations team actively participates in the institutional Course Evaluations Advisory Group, helping to shape the culture of assessment at the University of Toronto by sharing current research on course evaluations, trends, and encouraging a collaborative approach to improving evaluation administration and use of evaluation data.
CTSI has established itself as a leader, nationally and internationally, on course evaluations. In addition to hosting the 2018 international Course Evaluation Institute, over this past academic year CTSI carried out 22 external consultations (see page 43 for details). Course Evaluations team members provided in-depth consultations to members of the inaugural CTSI SoTL Cohort, lending expertise on research and assessment practices on projects of faculty members across U of T. As well, course evaluations are one source of data for teaching dossiers at the U of T, and over this past year CTSI team members supported individual faculty members and administrators across the institution in teaching dossier preparation. For consultation metrics see Appendix E.
CTSI hosted the inaugural Course Evaluation Institute, October 18-19, 2018, a user-led international gathering of course evaluation administrators seeking to build a collaborative and interactive community focused on course evaluations and teaching assessment. Over the two days, CTSI adopted a leadership role in fostering this growing community of practice.
“Working with CTSI has been an extremely enjoyable experience. The workshops I have attended as an instructor have really helped me connect with students with various abilities. As an administrator, I have benefited from many of CTSI’s documents. I have found the Course Evaluation Guidelines for Academic Administrators particularly helpful in chairing tenure committees. Overall, I find the staff at CTSI extremely helpful and responsive. Thanks for all the great help!”
Wendy Duff, Professor and Dean, Faculty of Information
“The CTSI expert team guided us through the University’s course evaluation framework. We felt supported to develop our divisional and program level items in our professional and graduate programs. Now that we have fully implemented the University’s course evaluation framework, we are able to utilize the data to enhance our Quality Assurances processes. For example, the information gathered from the course evaluations is now embedded in our annual reporting structure and helps to inform our curriculum reviews.”
Aleksandra Bjelajac Mejia, Assistant Professor, Teaching Stream/Interim Director, Professional Programs and Lachmi Singh, Director, Education Programs & Administrative Services, Faculty of Pharmacy