More than just a survey: student learning experience is at the core of the new U of T course evaluation framework
In keeping with its commitment to excellence in teaching, the University of Toronto is pleased to announce the launch of a new course evaluation framework. This new framework is aimed at assessing factors that contribute to students’ overall learning experiences in their courses. At the heart of this new framework are new core institutional course evaluation questions, which stem from the University’s expectation that students have valuable learning experiences in their courses. Stated simply, the University of Toronto expects that its courses are intellectually stimulating, create environments conducive to real learning, and provide opportunity for students to demonstrate their knowledge.
The need for a new course evaluation framework has been on the minds of the academic administration for some time. Recognizing the importance of an institutional approach to the assessment of students’ learning experiences in their courses, in 2009 a working group chaired by Professor Jill Matus, Vice Provost – Students, and Professor Edith Hillan, Vice Provost – Academic and Faculty Life, and consisting of academic representation from divisions across the institution, was created. After 2 years of extensive consultation with students, faculty, and staff, a rigorous review of the empirical literature on the assessment of teaching, and an examination of practices at other institutions, the working group recommended a number of significant changes to current course evaluation practice at the University of Toronto. Briefly, the Working Group recommended the development of a new university policy regarding the evaluation of University of Toronto courses (passed at Governing Council in 2011), the complete redesign of course evaluation questions that stem directly from the institution’s teaching and learning goals for students, a centralized team of experts to manage and to evaluate the framework, a framework that provides both summative and formative student feedback to instructors, and a new online system to meet the university’s commitment to student needs and campus sustainability. To monitor the realization of these recommendations, a Course Evaluation Implementation Group was established as well. Co-chaired by Professor Cheryl Regehr, Vice Provost – Academic Programs, and Professor Carol Rolheiser, Director, Centre for Teaching Support & Innovation, this group has been and continues to be instrumental in monitoring and advising each stage of implementation.
The new course evaluation framework is innovative in its approach to the evaluation of teaching. It balances the need to assess broad institutional expectations for teaching and learning while recognizing that students’ learning experiences vary by specific pedagogical goals set out by an academic discipline. Thus, beyond the core institutional questions, course evaluations are customizable, providing divisions, departments, and instructors the option to select their own questions from a new course evaluation question bank. This multi-level assessment approach ensures that students’ learning experiences are assessed broadly and within learning contexts. Course evaluation questions are both quantitative and qualitative, providing faculty with extensive information to incorporate into future course design or redesign.
The University of Toronto’s new course evaluation framework is also innovative in its involvement with students in both design and administration. Not only did students from across the institution play a central role in the actual wording of core institutional questions, students’ concerns regarding the lack of opportunity to provide thoughtful feedback about their learning experiences during class time was recognized as a key priority. To meet this concern, the University of Toronto licensed an online course evaluation delivery tool from Canadian company, eXplorance. Within the new framework, students now have approximately 2 weeks during the end of their courses to provide feedback from their personal computers or electronic devices. Students’ responses to the new questions and the online system have been overwhelmingly positive. Of a sample of 10,000 students surveyed who have used the new framework and system to evaluate their courses, approximately 60% felt that completing an evaluation using the new system was better than completing evaluations on paper in class.
The University of Toronto has invested in a number of resources to ensure the innovative goals of the new course evaluation framework are met. As a new centralized service, the course evaluation framework is managed through the Centre for Teaching Support & Innovation, under the direction of the Associate Director, Dr. Pam Gravestock and a new Course Evaluation Support Officer, Dr. Cherie Werhun. In addition to managing the online system, course evaluation reports, and the course evaluation item bank, the Centre for Teaching Support & Innovation is responsible for ongoing analysis of the new framework and for working with divisions to create course evaluation divisional procedures, interpretation guidelines, and educational materials to assist academic administration, faculty, and students in their use of course evaluation data.
The implementation of the new course evaluation framework has expanded considerably since its inception in the Fall of 2011. To date, approximately 1000 courses have been evaluated, generating over 16,000 evaluations within Arts & Science, Nursing, UTM, and UTSC. This fall, 2012, Arts & Science and Social Work will be participating fully in the new framework, and other divisions are set to join in 2013. As additional faculties join the new course evaluation framework, the University of Toronto is committed to meeting this growth with new support. This fall, course evaluations will be integrated with Blackboard and in 2013, a new U of T course evaluation app will be released for use with students’ iPhones.
The implementation of the new course evaluation framework at the University of Toronto has been no small task, and has involved the coordinated investment of numerous resources, faculty, staff, and students. Overall, this campus-wide dedication to the new course evaluation framework at the University of Toronto suggests that the institution is committed not only to excellence in teaching but to excellence in the evaluation of teaching, as well.
It is indeed an exciting time for teaching at U of T!
For more information on the new course evaluation framework and online system, please visit: http://www.teaching.utoronto.ca/teaching/essentialinformation/evaluation-framework.htm and http://www.courseevaluations.utoronto.ca/
For instructors who are interested in promoting the new course evaluation initiative in their classroom, Just-in-Time Powerpoint slides are available from the CTSI website.
