I am a graduate student at Elite State University, finishing my first year working as a TA. Following my first semester, I thought I had done a fairly good job teaching the undergrads; they seemed more or less happy in discussion section, and appreciated the extensive comments I put on their assignments. Then I got the student evaluations. For a couple of days I was rather bummed out (a sample comments: "He's kind of a dick in his emails").
After reading through Rate Your Students, though, I realized the similarity of my experience to other college teachers and learned to take the evaluations for what they are (fairly meaningless).
I'm still surprised that this crude, inaccurate "tool" is used in actual hiring decisions. There is a pretty clear pattern among my students: if they did well in the class, I was a good TA; if they did poorly, I was a bad TA. Not to beat a dead horse, but the student-as-consumer mentality is clearly present in my experience. There is a sense of entitlement; if I show up to most classes, turn in my assignments, and do some of the reading, clear I deserve no less than a B+!
I find it hard to believe that college administrators (let alone tenured faculty members on the tenure committees) don't understand this. So why can't we do some simple statistics for student evaluations, if they are going to be considered in tenure decisions? It's not that difficult.
The null hypothesis is that there is a positive correlation with grade and rating: as the grade increases, the evaluation of the teacher will increase. We can easily maintain the anonymity of student evaluations by simply having them enter their ID number into the evaluation. Their final grade will then be associated with their evaluation. If the pattern holds up (and we can't reject the null hypothesis), the evaluations are a wash. They don't tell us anything. But deviations from the predicted pattern could indicate something relatively significant. Professors who have students who receive lower grades but still get high marks on evaluations are probably fairly good at teaching. Similarly, professors who have students with high grades giving low marks on evaluations probably are not good at teaching. This seems so obvious that I shouldn't even have to describe this method.
But if junior faculty have to worry about student evaluations for their tenure decisions, we should recognize the relative arbitrariness of the evaluations. Such important decisions should not be based on 18-year-olds who were the big fish in the little pond in high school but find out in their first semester that they actually have to work to earn a grade, and thus take out their frustrations on evaluations.
