The Agony and the Ecstasy of Student Teaching Evaluations

Opening up student evaluations—They are the best of times, they are the worst of times (with all due respect to Dickens).

Before new semesters amble through the starting gates, it is a good time to put student evaluations of teaching (SETs) from preceding semesters in their right perspective. I have been reading and thinking a lot about evaluations recently (especially their relationship to learning, see Stroebe, 2016) so what better time to pull together many different issues relating to student evaluations.

One on hand, I hope to see in SETs, some evidence of the effort I pour into teaching, of a recognition of the thoughtfulness inherent in designing a course well, of growth and learning coming from rising to the intellectual challenges I endeavor to provide, and of my attempts to be organized, knowledgeable, and passionate for my material.

On the other hand, I know there may also be evidence that I may not have sufficiently matched or met every students’ needs, that my personality, style, delivery or pedagogical decisions may have failed to create the learning environment I hoped for, or that some student may just used the evaluation to process a bad day and stressful time of the semester. Oh the joys of reading student evaluations.

I know some faculty who cannot help but focus on the (often few) negative comments they receive. I know others who immediately post their most complimentary comments to social media. One person I know claims to not even open the evaluations, transferring them instead, directly to the wastebasket (gosh, I hope he at least recycles).

SETs have been studied, praised, and criticized for many years now. There are strong advocates for them (Boysen, 2016), large studies showing their effectiveness (e.g., Spooren, Brockx, & Mortelmans, 2013), and others decrying their shortcomings (e.g., Clayson, 2009). Different experts have different favorite instruments (e.g., the SEEQ, Marsh, 1982), and some even suggest abolishing SETs completely.

My read of the pro and con literature suggests a valid SET has utility. Whereas we can argue about the validity of SETs, and easily agree that SETs should not be the sole measure of teaching effectiveness, we should not underestimate its potential benefits to the individual teacher as a form of professional development, Chairs and Deans for use in promotion, merit, and tenure, and Colleges as a whole when used as part of accreditation.

For some schools SETs are the easy way out. Colleges need to pay more attention to how teaching is evaluated with multiple measures mirroring its commitment to excellent teaching. Pragmatic steps include using a multidimensional, valid and reliable SET instrument as one form of evidence of teaching effectiveness, supporting college wide formative assessment, faculty development opportunities to foster the creation of teaching portfolios (syllabi, examples of assessment, teaching philosophy), a mechanism for peer observations of teaching by impartial third parties, and a strong, explicit mentoring system especially targeting first and second year faculty to shape a discussion of SETs.

A good SET provides BOTH quantitative and qualitative information, is multidimensional to allow the separate evaluation of the course from the instructor as well as different components of effective instruction, allows individual instructors to add questions of specific interest to them, and is readily available to faculty, their mentors, and key administrators as needed (e.g., Chairs, Deans), all of whom should be educated in best practices of SET interpretation (see Boysen, 2015).

Some findings to note:

• Small class sizes and outliers (extreme student ratings) can skew averages.
• Frequency scores and distributions are key in SETs, do not over emphasize means.
• Students may have biases towards female instructors and instructors of color which need to be acknowledged and factored into any review.
• Although online completion shows low response rates, having students complete them online in class should help.
• Faculty need to be committed to the value of their SET and convey this to students (something both students and faculty mentioned is not happening).
• Let’s not forget many factors may contribute to a particular student’s evaluation of you.

Want more on SETs? Run, don’t walk to Boysen (2016), for a great piece that nicely summarizes the literature on SETs and provides further recommendations for selection and use.

Although sometimes bittersweet, I look forward to getting my evaluations. Faculty should take time to reflect on the comments and scores and make them count. If comments are invalid ignore them, but if they are worthwhile, CHANGE.

If we faculty are not open to changing what we do when evaluations suggest issues, we are doing our craft and our students a disservice.

References

Boysen, G. A. (2015). Uses and misuses of student evaluations of teaching: The interpretation of differences in teaching evaluation means irrespective of statistical information. Teaching of Psychology, 42, 109-118. 

Boysen, G. A. (2016). Using student evaluations to improve teaching: Evidence-Based Recommendations. Scholarship of Teaching and Learning, 2, 273-284.

Clayson, D. E. (2009). Student evaluations of teaching: Are they related to what students learn?: A meta-analysis and review of the literature. Journal of Marketing Education, 31, 16-30.

Marsh, H. W. (1982). SEEQ: A reliable, valid, and useful instrument for collecting students’ evaluations of university teaching. British Journal Of Educational Psychology, 52(1), 77-95.

Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83, 598-642. 

Stroebe, W. (2016). Why good teaching evaluations may reward bad teaching: On grade inflation and other unintended consequences of student evaluations. Perspectives on Psychological Science, 11.