The evaluations get worse - Macleans.ca

The evaluations get worse

There is no good idea that can’t be killed by administrators and governments

by

Last year, around this time, I found myself complaining about student evaluations. Not what the students actually said, mind you, the very idea of them. Well, it’s that time of year again, and my general view of the situation has not improved. And as for the specifics, things have become decidedly worse. Let me explain.

It used to be the case here at CBU that individual departments could create their own course evaluation forms, provided they were approved by the Dean. In fact, strictly speaking, individual faculty members could create their own forms, again, if the Dean signed off on them. Such a system did not fix the many problems inherent in student evaluations, but it helped mitigate them by tailoring the form to the discipline and by providing individual faculty with more input into their own department’s evaluation process.

In university life however, there is no good idea that can’t be killed by administrators and governments. Our administration, you see, hated the old system since it was too complicated and made for a lot of work for secretaries. Still worse, the Maritime Provinces Higher Education Commission, one of whose functions is to give the public the impression that they are keeping a close watch on the quality of university instruction in the three maritime provinces (yes, three, Newfoundland is not part of the Maritimes), strongly suggested that we have one standard evaluation form to be used throughout the entire institution. Their idea is that the new instrument will allow for better comparison of instruction across the whole range of departments and schools. This, of course, ignores the fact that allowing for comparisons is useless when the data is meaningless, and in fact, the new administrator and civil servant friendly form is so generically bureaucratic, that no real accountability could possibly come from it.

Some of the questions (or rather statements with which students agree or disagree) are not terribly wrong headed, just anodyne.  For instance:

The instructor spoke clearly  and audibly.

This one has a bit of a trap in it, since, from my experience, students seem unusually sensitive to English spoken with a foreign accent, and will claim not to be able to understand a prof whose accent is noticeable, even if said professor’s English is perfectly good. But apart from that issue, the question relates to a minor mechanical aspect of teaching, and making it one of twenty or so identically formatted questions gives the impression that it is just as important as more important things. You can almost hear the tenure committee debate: “Well Professor Facile has not completed his doctorate, nor has he published any articles, nor is his teaching innovative, but on the other hand his students say he speaks so clearly!” I’m not saying this type of question should not be included, but technical matters like this should be set apart from more substantial ones. Or they should be somehow weighted appropriately.

Other items are so vague that it’s hard to see how students could reasonably evaluate them, and harder to see how instructors could improve on them even if they wanted to. For instance:

The instructor showed a genuine concern for students’ progress and was approachable.

It’s not at all clear to me what is meant by “genuine concern” here. Is the instructor supposed to phone students at home and ask how they are doing? Fortunately, there is no question that says “My instructor creeped the hell out of me.” And how would a student distinguish genuine concern  from fake concern? I have a feeling that many students will take low grades and high standards as evidence of a lack of “concern” on the part of the instructor. After all, if Professor Highbar was so concerned about my success, he would have passed me, right?

And what does “approachable” mean? Many people find intelligence intimidating — does that mean we should avoid hiring very intelligent professors for fear that they will seem unapproachable? Are women usually more approachable than men? And how would a professor who scores poorly on that question aim to be more approachable? I have known of professors who have been so approachable that they have become close friends and confidants with their students, to the point of being unprofessional. You can’t fairly grade the papers of your new BFF.

It just gets worse after this. I mean look at some of these:

This course has improved my critical reading and writing skills.

The readings were useful in achieving the goals of this course.

The instructor was fair in measuring student performance.

All of these questions are dubious because they ask students to evaluate things they are in no position to evaluate. It would be like evaluating physicians by asking patients if they were happy with their diagnoses. But because so much weight is placed on these evaluations, professors have a strong interest in pandering to them, nonetheless. And yet,unless we are very naive, we must acknowledge that students who rate their professor’s grading as not fair are really complaining about their grades being too low. How many students who get an A think the grade was unfair? How many students who get an F think the grade was just right?

None of this would matter if it didn’t matter. But when Professor Noobi sees that her students say the readings in her course are not “useful,” she knows that she can more easily secure her position and livelihood by reducing or eliminating those readings, whether or not they were intellectually justified. Good professors, of course, resist such temptations, but when one’s career is at stake, small compromises (perhaps made even unconsciously) are all too easy, and year after year, they add up. Or rather, drag down. In any case, professors simply should not be put into such positions.

The sad part is that with a little courage and creativity, evaluations could be made to be, at least, somewhat useful. When this new form was in its draft stages, I actually suggested to the committee a large number of questions. One of them was:

My instructor was funny.

That question was rejected on the grounds that not everyone is naturally funny. But what about those of us who are not naturally approachable? Here are some other questions I suggested that did not make the cut:

The instructor seemed to hold students to high standards.

The instructor treated students as though they were responsible for their own success in the course.

The instructor stressed thinking skills, not just knowledge of material.

Notice, by the way, that that last one is not the same as the one about “critical reading and writing” I criticized above. Students can fairly comment on what the instructor seemed to stress, but what they learned may not have anything to do with the quality of the instruction provided. Here’s another good example:

The instructor encouraged students to see how complex the issues raised in the course were.

I like this question because often evaluations ask students about whether the instructor is clear, and students often praise their instructors for making “everything clear and easy to understand.” But I worry that clear and easy to understand is really the student’s perception of what a professor might consider leaving out the hard parts and  glossing over the difficulties. If a university course is well done, there should be a lot of things that are decidedly not easy to understand.

As I said, none of these suggestions was accepted by the committee who created the new list, and to be fair, I’m sure they were trying to balance out suggestions from all sides, so it’s really no wonder that the result is what it is: outrageous only in its timidity, and offensive only in its extreme inoffensiveness. But what is the solution? Let people create their own forms, provided they are approved by the Dean?

Oh, right. That’s what we used to have.