Student Ratings of Instruction System

• Advocate for the replacement of an aged series of SRI questions with a more modern, pedagogically relevant, established, and evidence-based series of hierarchical SRI question framework to a reluctant group of stakeholders.

• Implement a robust digital enterprise SRI system to replace an antiquated largely paper-based system to be used by the same group of stakeholders.

• Contend with the effects of a rapid COVID-19 remote teaching mandate put into place for the entirety of the implementation.

• Create contingency plans across 2 faculty strikes that occurred during the implementation.

The Centre for the Advancement of Teaching and Learning

• Research, Evaluation, and Innovation Team (me)

• Learning Technologies and Solutions Team

• Operations Team

Information Services and Technology Department

Office of the Provost and Vice-President (Academic)

Evaluation of Teaching and Courses Review Subcommittee, Senate Committee on Instruction and Evaluation

Marketing Communications Office

Explorance, Enterprise Technology Solution Provider

The project sought to:

• Replace an aged series of SRI questions, with a more modern, pedagogically relevant, and evidence-based hierarchical series of SRI question framework.

• Transition from an antiquated and largely paper-based method of SRIs to a streamlined digital method.

• Facilitate awareness and advocate for the changes made to the SRI system, as well as to provide support for and develop individualized training material for all deans, department heads, instructors, SRI co-ordinators, and students.

Throughout the project a number of research methods were utilized:

• Secondary Research

• Focus Groups

• Usability Testing

• Surveys

• Interviews

Results

• Successfully advocated for and led to the implementation of pedagogically relevant evidence-based series of hierarchical SRI questions incorporating multiple stakeholders at different levels:

  • Institutional core questions (mandatory)

  • Instructor questions

  • Departmental questions

  • Faculty questions

• Replaced an antiquated largely paper-based SRI method method with a streamlined digital method incorporating an abundance of added features that were either not possible or very resource intensive through the paper-based method.

  • Maintained student completion rate of SRIs at around 30% (the average at post-secondary institutions) throughout the transition from paper-based to digital SRI system (this value typically decreases when digital SRIs are introduced, a fear that administration held), due in part to our instructor awareness efforts.

Impact

• Introduced powerful online SRI reporting and data analysis features that can be used across instructors, departments, faculties, and institutionally. Among other important initiatives, the added features will provide guidance toward programming and professional development, ultimately leading to improved instruction and student academic outcomes at the University.

• Eliminated the use of +50,000 lbs. of paper per year, manual input of 240,000+ paper SRI forms per year, and the long-term storage of all SRI forms in Libraries.

• A rapid review and subsequent systematic review was conducted to address concerns raised by faculty in terms of potential biases present in SRIs. This led to a publication in the peer-review journal, Canadian Journal of Educational Administration and Policy:

  • Stoesz, B. M., De Jaeger, A., Quesnel, M., Bhojwani, D., & Los, R. (accepted). Bias in Student Ratings of Instruction: A Systematic Review of Research from 2012 to 2021. Canadian Journal of Educational Administration and Policy.