6

The evaluations get worse

There is no good idea that can’t be killed by administrators and governments


 

Last year, around this time, I found myself complaining about student evaluations. Not what the students actually said, mind you, the very idea of them. Well, it’s that time of year again, and my general view of the situation has not improved. And as for the specifics, things have become decidedly worse. Let me explain.

It used to be the case here at CBU that individual departments could create their own course evaluation forms, provided they were approved by the Dean. In fact, strictly speaking, individual faculty members could create their own forms, again, if the Dean signed off on them. Such a system did not fix the many problems inherent in student evaluations, but it helped mitigate them by tailoring the form to the discipline and by providing individual faculty with more input into their own department’s evaluation process.

In university life however, there is no good idea that can’t be killed by administrators and governments. Our administration, you see, hated the old system since it was too complicated and made for a lot of work for secretaries. Still worse, the Maritime Provinces Higher Education Commission, one of whose functions is to give the public the impression that they are keeping a close watch on the quality of university instruction in the three maritime provinces (yes, three, Newfoundland is not part of the Maritimes), strongly suggested that we have one standard evaluation form to be used throughout the entire institution. Their idea is that the new instrument will allow for better comparison of instruction across the whole range of departments and schools. This, of course, ignores the fact that allowing for comparisons is useless when the data is meaningless, and in fact, the new administrator and civil servant friendly form is so generically bureaucratic, that no real accountability could possibly come from it.

Some of the questions (or rather statements with which students agree or disagree) are not terribly wrong headed, just anodyne.  For instance:

The instructor spoke clearly  and audibly.

This one has a bit of a trap in it, since, from my experience, students seem unusually sensitive to English spoken with a foreign accent, and will claim not to be able to understand a prof whose accent is noticeable, even if said professor’s English is perfectly good. But apart from that issue, the question relates to a minor mechanical aspect of teaching, and making it one of twenty or so identically formatted questions gives the impression that it is just as important as more important things. You can almost hear the tenure committee debate: “Well Professor Facile has not completed his doctorate, nor has he published any articles, nor is his teaching innovative, but on the other hand his students say he speaks so clearly!” I’m not saying this type of question should not be included, but technical matters like this should be set apart from more substantial ones. Or they should be somehow weighted appropriately.

Other items are so vague that it’s hard to see how students could reasonably evaluate them, and harder to see how instructors could improve on them even if they wanted to. For instance:

The instructor showed a genuine concern for students’ progress and was approachable.

It’s not at all clear to me what is meant by “genuine concern” here. Is the instructor supposed to phone students at home and ask how they are doing? Fortunately, there is no question that says “My instructor creeped the hell out of me.” And how would a student distinguish genuine concern  from fake concern? I have a feeling that many students will take low grades and high standards as evidence of a lack of “concern” on the part of the instructor. After all, if Professor Highbar was so concerned about my success, he would have passed me, right?

And what does “approachable” mean? Many people find intelligence intimidating — does that mean we should avoid hiring very intelligent professors for fear that they will seem unapproachable? Are women usually more approachable than men? And how would a professor who scores poorly on that question aim to be more approachable? I have known of professors who have been so approachable that they have become close friends and confidants with their students, to the point of being unprofessional. You can’t fairly grade the papers of your new BFF.

It just gets worse after this. I mean look at some of these:

This course has improved my critical reading and writing skills.

The readings were useful in achieving the goals of this course.

The instructor was fair in measuring student performance.

All of these questions are dubious because they ask students to evaluate things they are in no position to evaluate. It would be like evaluating physicians by asking patients if they were happy with their diagnoses. But because so much weight is placed on these evaluations, professors have a strong interest in pandering to them, nonetheless. And yet,unless we are very naive, we must acknowledge that students who rate their professor’s grading as not fair are really complaining about their grades being too low. How many students who get an A think the grade was unfair? How many students who get an F think the grade was just right?

None of this would matter if it didn’t matter. But when Professor Noobi sees that her students say the readings in her course are not “useful,” she knows that she can more easily secure her position and livelihood by reducing or eliminating those readings, whether or not they were intellectually justified. Good professors, of course, resist such temptations, but when one’s career is at stake, small compromises (perhaps made even unconsciously) are all too easy, and year after year, they add up. Or rather, drag down. In any case, professors simply should not be put into such positions.

The sad part is that with a little courage and creativity, evaluations could be made to be, at least, somewhat useful. When this new form was in its draft stages, I actually suggested to the committee a large number of questions. One of them was:

My instructor was funny.

That question was rejected on the grounds that not everyone is naturally funny. But what about those of us who are not naturally approachable? Here are some other questions I suggested that did not make the cut:

The instructor seemed to hold students to high standards.

The instructor treated students as though they were responsible for their own success in the course.

The instructor stressed thinking skills, not just knowledge of material.

Notice, by the way, that that last one is not the same as the one about “critical reading and writing” I criticized above. Students can fairly comment on what the instructor seemed to stress, but what they learned may not have anything to do with the quality of the instruction provided. Here’s another good example:

The instructor encouraged students to see how complex the issues raised in the course were.

I like this question because often evaluations ask students about whether the instructor is clear, and students often praise their instructors for making “everything clear and easy to understand.” But I worry that clear and easy to understand is really the student’s perception of what a professor might consider leaving out the hard parts and  glossing over the difficulties. If a university course is well done, there should be a lot of things that are decidedly not easy to understand.

As I said, none of these suggestions was accepted by the committee who created the new list, and to be fair, I’m sure they were trying to balance out suggestions from all sides, so it’s really no wonder that the result is what it is: outrageous only in its timidity, and offensive only in its extreme inoffensiveness. But what is the solution? Let people create their own forms, provided they are approved by the Dean?

Oh, right. That’s what we used to have.


 

The evaluations get worse

  1. Speaking from the side of a student who began their university career within the walls of CBU, I enjoyed filling out the previous evaluation forms. It gave me an opportunity to provide constructive feedback to professors whose courses I thought needed improvement and to pay compliment where one was earned. After graduating from CBU, I moved onto a university using standradized “scantron” type evaluations. My level of attention and care excercised towards the evaluations significantly decreased because they were both vague and were not specific. Some professors continued to give out an additional evaluation form specific to their course which I appreciated.

  2. At UVic we have a standardized evaluation form as well, and I’ve always found it useful. Now I’m not sure what subjects you teach, but if you think that your students can’t answer:

    “The readings were useful in achieving the goals of this course”

    Then perhaps you haven’t done an adequate job of explaining the goals of the course in your course syllabus. I’m in a combined social sciences / fine arts program and I’ve found that in most cases the professor has done a decent job of articulating the goals and learning outcomes of the course.

  3. Matt, part of the problem may be that I teach English literature where the readings are so tied up with the goals of the course that the question of their usefulness barely makes sense.

    Do I really have to explain to students in a Shakespeare class why reading Hamlet is relevant to the learning outcomes of the course?

    I concede that the question may have more relevance in some other disciplines, but then, that just highlights the underlying problem: the one-size-must-fit-all evaluation form.

  4. I have to agree that these generic forms are pretty useless. Ones that I’ve seen recently include “the textbook was useful” in a course with no required textbook, and the aforementioned one about readings, in a course with no readings! Or “I am motivated to learn more about these subject areas.” How does that question have anything to do with the instructor? Goodness knows I’ve had to take courses that don’t interest me at all, just to fulfill my program requires, so no, I’m not interested in learning more about that subject. That prof may have been excellent, but if I don’t have an interest in the subject area, I’m not motivated to learn more.

  5. The instructor encouraged students to see how complex the issues raised in the course were. -> The instructor encouraged students to see how the issues raised in the course were complex.

    Can’t help myself.

  6. Standardized forms, or any standardization for that matter, usually is imposed because some significant proportion of users do a very poor job when designing their own form (or process). To avoid these substandard products, a minimum standard is imposed. The competent performers pay the price of lost flexibility but the cause of the problem is not really administrators trying to make their staff’s life difficult for no good reason. The suggestion that an administrator might have the time to approve every instructor’s individual evaluation form is clearly not realistic. I suspect the unstated concern here is that professors do not like to be evaluated.

Sign in to comment.