Student Evaluations of Faculty (SEF)

Schoolboy

Receiving

Bare Bottom Birching

Medieval Source

(Medieval! Should I update my pedagogical methods?)

(Image from Wikipedia)

Readers will recall my previous two posts on student evaluations of teaching, both in response to Professor Stanley Fish's two columns on the topic. In the latter of these two posts, I noted Professor Fish's reference to "the preponderance of studies document[ing] . . . [the] non-correlation" . . . "between student evaluations and effective teaching ," and I wondered:

Professor Fish's column is nonacademic and thus cites no sources, but I'm curious about the studies alluded to, for I would like to have cited such studies in previous years when issues of this sort arose in discussions over good teaching.

Well, in response to my query, an instructor who read my post emailed me some studies and papers. I've only had time to skim most of them (one is 50 pages long!), but a couple caught my attention. A paper titled "Student Evaluations: A Critical Review," written by philosophy professor Michael Huemer (University of Colorado, Boulder), states the following concerning student evaluations of faculty (SEF) -- and I've also included his sources cited in the passage:

The most common criticism of SEF seems to be that SEF are biased, in that students tend to give higher ratings when they expect higher grades in the course. This correlation is well-established, and is of comparable magnitude, perhaps larger, to the magnitude of the correlation between student ratings and student learning (as measured by tests) . . . . Thus, SEF seem to be as much a measure of an instructor's leniency in grading as they are of teaching effectiveness. The correlation holds both between students in a given class and between classes. It also holds between classes taught by the same instructor, when the instructor varies the grade distribution. And it affects ratings of all aspects of the instructor and the course. (6) Many believe that this causes rampant grade inflation. (7)

6. See Rice, 335-6; Wilson; Greenwald and Gillmore, 1214.

7. See Goldman; Sacks.

Goldman, Louis. "The Betrayal of the Gatekeepers: Grade Inflation," Journal of General Education 37 (1985): 97-121.

Greenwald, Anthony G. and Gerald M. Gillmore. "Grading Leniency Is a Removable Contaminant of Student Ratings," American Psychologist 11 (1997): 1209-17.

Rice, Lee. "Student Evaluation of Teaching: Problems and Prospects," Teaching Philosophy 11 (1988): 329-44.

Sacks, Peter. Generation X Goes to College (LaSalle, IL: Open Court, 1986).

I'm not sure whether or not Professor Heumer has published this paper, but it can be read online via his website.

The other article that caught my attention is a published paper, "No pain, no gain? The importance of measuring course workload in student ratings of instruction," Journal of Educational Psychology, Volume 89, Nr. 4 (1997), pages 743-751, by Anthony G. Greenwald and Gerald M. Gillmore. Here's the abstract:

Samples of about 200 undergraduate courses were investigated in each of 3 consecutive academic terms. Course survey forms assessed evaluative ratings, expected grades, and course workloads. A covariance structure model was developed in exploratory fashion for the 1st term's data, and then successfully cross-validated in each of the next 2 terms. The 2 major features of the successful model were that (a) courses that gave higher grades were better liked (a positive path from expected grades to evaluative ratings), and (b) courses that gave higher grades had lighter workloads (a negative relation between expected grades and workload). These findings support the conclusion that instructors ' grading leniency influences ratings. This effect of grading leniency also importantly qualifies the standard interpretation that student ratings are relatively pure indicators of instructional quality.

Both papers confirm something that I've long wondered about and also help me to analyze a set of course evaluations that I received yesterday from students in a particular department where the students are generally quite good in English due to having attended international schools or having been educated overseas. This set of evaluations was lower than I would ever have wanted. Indeed, I had expected outstandingly high marks based on classroom rapport and the depth of what I had taught the students about researched essay composition. I received, however, surprisingly low marks. When I checked the written comments, I found complaints that they, as students of their particular department, should be required to take essay composition courses since they are already good in English and had already learned how to write essays in high school.

Think about that. The set of evaluations that I received for that course were low because the students thought that they had nothing to learn, didn't want to take the course, and were annoyed by the requirement.

Such an attitude might be more acceptable to discover if these students of a particular department really did have nothing to gain from the course, but their view of their own abilities was grossly exaggerated. All of them had difficulties in reasoning soundly and using evidence effectively, and all of them needed to learn how to do research well and cite sources properly. The grades that they expected to receive reflected these weaknesses.

Let me explain that last point. During the semester, and in every course, I grade on an absolute scale based on what I consider to be rigorous standards. The grades that students receive during the semester are thus significantly lower than what they are accustomed to receiving, based on many other courses that they have taken in their major fields. Students often come to see me and say that they have never received such low grades before . . . and they are generally skeptical that they deserve such grades. This is especially the case with students in the particular department noted above.

Based on the two scholarly papers cited in today's post, I infer that my undeservedly low evaluations by students of this particular department were due to their expectation of low final grades for the course. The students' comments didn't mention this expectation, but such is unsurprising since that would be to acknowledge that they weren't as good as they thought themselves to be.

The irony is that despite their expectations, these students received good grades. Why? Because while I grade on the absolute scale during the semester, I adjust to a high curve for the final grades. I do this adjustment because we are required to grade on this sort of curve. I would prefer to assign grades on an absolute scale, but I can't. Nevertheless, because I have always felt that students should have a more realistic appraisal of their actual performance, I've chosen to rank them on an absolute scale up until I finally assign course grades.

I guess I'd better stop that pedagogical practice and grade on a relative scale throughout the semester unless I want to keep getting low evaluations from students in that particular department.

For the record, I was reasonably satisfied with the other three course evaluations that I saw yesterday. One of them was lower than I wanted but about what I had expected. The students of that course had very low English skills, and I never quite figured out the best way to help them . . . but I'll keep working on finding a way.