It's that time of year again, when the administration sends out its minions with their student opinion surveys. I hesitate to use the term "course evaluation." Now I'm sure there are a bunch of yahoos out there who claim that their instrument is valid and reliable and is currently being used in however many institutions. And of course that may be the case. I have no problem with a well-designed and well-tested measurement instrument. Where my beef emerges is around the idea that that students are capable of - hell, even understanding - what evaluation entails.
I'm not really sure that the bulk of them are in a position to do so. Let me give you an example - something from the trenches: I ask students to write weekly annotated bibliographies. A component of the assignment is to write a one or two line evaluation of the reading. And I make it clear that I'm looking for an intellectual or academic evaluation - as in something like: "The author raises some provoking questions about X, but does not address issue Y." Or something like "This article offered an interesting contrast to the previous two articles we've read given that it adopted a different methodology and a different theoretical framework." 
Not surprisingly, there is seldom an intellectual or academic evaluation to be found. Rather, I get drivel like the oh so vapid, "I liked the article," or the trenchant "The article was too long." Or, "The author uses too many example." Or my personal favorites, "The article has too many big words," and "This article was too hard."
Now if I can't get more than half of my students to offer an intellectual evaluation of an assigned reading - and I've given them plenty of examples of what an evaluation ought to look like - how am I supposed to take their "course evaluation" seriously? I have students who don't know what "pedagogy" means, what a learning objective looks like, and 25% of them haven't read the fucking course outline.
Evaluation assumes that one has some knowledge about what is being evaluated. What the hell do they know about teaching? You're asking a 20 year old to critically evaluate and reflect on my capacity to teach? When was the last time a 20 year old read up on how to construct fair and accurate multiple choice questions on the weekend so that he could ensure that the exam he was giving the following week was well constructed? What 20 year old knows about Bloom's taxonomy?
I'm not claiming that what students say about their educational experiences can't be useful or shouldn't be solicited. Their feedback is, I think, productive and can go a long way in assisting us in improving our efforts at teaching. Feedback is, I think, quite a different ball of wax when compared to "evaluation" and I'm more than willing to entertain student feedback. The problem is, "course evaluations" don't allow for this - given that they are, among other things, summative rather than formative assessments.
So I have a question - one asked in the interest of doing what we can to help our students become the adults we all know they can be. What can we do to generate pedagogically useful feedback from your students?
