BY TAMMIE CUMMING, M. DAVID MILLER, FREDRIK DEBOER, AND JENNIFER BERGERON
Photo by Bongkarn Thanyakij/Pexels.
It seems that almost no aspect of academic life has been untouched by the coronavirus pandemic and the drastic measures governments and institutions have taken to respond to it. Nearly every postsecondary institution has adopted distance learning, forcing many faculty and students to adapt to a new world in an instant. Beloved collegiate traditions like commencement and spring break have been canceled or postponed. Some students and faculty members have argued in favor of switching to a pass-fail system, and many institutions will at least offer pass-fail options. These developments have played out in real time as administrators and faculty members struggle to adjust to such dynamic and unpredictable conditions.
One salient question that has arisen concerns a practice common to most institutions, a practice that causes some to cheer and some to groan: the administration and completion of student evaluation or perception of teaching surveys. Though these surveys can be viewed as a chore by students and as a reductive measure of teaching quality by faculty, student input is entrenched in the evaluation of faculty for annual review, promotion, and tenure decisions.
Typically, surveys are administered toward the end of a given semester, with instructors waiting outside for in-class administration methods so that students can feel free to complete them in privacy. Other student evaluation systems are now entirely online, with students encouraged by automated emails from administration to complete them—and sometimes incentivized to complete them in order to receive their grades a little earlier at the end of the semester. In either case, students will typically be called on to rate their instructors on a variety of dimensions along ordinal scales, such as “The instructor had deep knowledge of the material—choose 1 for strongly disagree to 5 for strongly agree.” In addition, these instruments typically allow for students to write in their own commentary.
Certainly the technology exists to conduct an online administration of the student evaluations of teaching no matter which type of survey administration mode was used pre-COVID-19. The question we should ask in the shadow of this pandemic and an abrupt transition to an online learning environment is not whether we can still implement student evaluations of teaching, but whether we should.
Student evaluations of teaching have been controversial for a long time. Many faculty members have long argued that the scales and categories used in these instruments are reductive and arbitrary, failing to capture the complexity and nuance that are inherent to the student-teacher relationship. Others have pointed out that student evaluations of teaching create an incentive for teachers to grade more leniently, as students earning high scores may rate a teacher more highly. And there is empirical research that suggests that these instruments may not have been developed with the key measurement principles in mind: validity, reliability, and fairness.
While these issues are an important part of the conversation, they are not our focus here. Instead, we advocate against administering student evaluations this semester for a more direct reason: the majority of faculty and instructors are teaching classes that are substantially different from the ones that they planned and prepared for. And students are evaluating faculty members in new contexts; lack of reliable access to the internet, familial duties or problems, and financial hardship are among the challenges that will potentially have a negative effect on ratings even though they are not the fault of instructors.
Instructors spend considerable time devising their lesson plans, materials, and assessments or tests. While some courses will be more or less conducive to moving online easily than others, all in-person classes suddenly transferred to an online learning environment will differ substantially from the course as originally envisioned by an instructor. Trips to the library, breakout sessions, small group work, hands-on demonstrations, and teaching physical procedures such as those typically conducted in a lab setting—among other techniques—can be difficult to replicate online without a significant amount of planning. This does not mean that the students will not get the instruction required to attain the course learning objectives, but they will now experience a different delivery of instruction.
If we evaluate instructors on classes that are substantially different from the ones they planned for, we risk evaluating them under unfair circumstances entirely beyond their control. And let’s not forget that it would be questionable at best to simply take the survey items used for in-person courses and transfer the same set of items to an exclusively online learning environment. The question of validity would certainly be up for debate.
From the standpoint of fairness to instructors and a simple desire to avoid biased evaluation results without evidence of validity, colleges and universities should carefully consider suspending student evaluations of teachers for the spring 2020 semester—and, depending upon the length of the pandemic, perhaps longer. If this teaching environment is prolonged, institutions should consider revising their current evaluations of teaching or developing new ones to pilot and evaluate.
There are many pressing issues on the minds of our faculty and our administrators surrounding the pandemic, but policy makers and administrators will need to review their current promotion and tenure policies, peer observation requirements, and other relevant issues within the context of the sudden shift to this online learning environment to ensure faculty are assessed and evaluated fairly.
Guest blogger Tammie Cumming is assistant vice president and associate provost at Brooklyn College of the City University of New York. M. David Miller is a professor of research and evaluation methods at the University of Florida. Fredrick DeBoer is assessment manager at Brooklyn College of the City University of New York. Jennifer Bergeron is director of educational research and evaluation at Harvard University.
What does the “covid” narratology have to do with student performance and evaluation considerations? They shouldn’t be suspended; the should be enforced–perhaps reinforced. Winners will adapt and succeed; indeed some will significantly outperform and lead; others will under-perform, and fail. That is the necessary calculus. What you recommend is yet more coddling. Necessary heightened performance evaluation and quality control is bi-directional and includes–perhaps to a greater extent–the academy itself.
I am well aware that there are pros and cons about the practice and subsequent use of student evals, I want to make a few points about the above essay:
1. The wording of this sentence — “Though these surveys can be viewed as a chore by students and as a reductive measure of teaching quality by faculty, student input is entrenched in the evaluation of faculty for annual review, promotion, and tenure decisions” — suggests to me an inherent bias against the practice, even in less contagious times.
2. A longer (biased) paragraph goes on to point out that “Student evaluations of teaching have been controversial for a long time. Many faculty members have long argued that the scales and categories used in these instruments are reductive and arbitrary, failing to capture the complexity and nuance that are inherent to the student-teacher relationship. Others have pointed out that student evaluations of teaching create an incentive for teachers to grade more leniently, as students earning high scores may rate a teacher more highly. And there is empirical research that suggests that these instruments may not have been developed with the key measurement principles in mind: validity, reliability, and fairness.”
Of course, a prolepsis is then added: “While these issues are an important part of the conversation, they are NOT our focus here.” If that is NOT your focus, why add this long critique of the evaluation process? maybe to prejudice the reader in advance, a common rhetorical device?
3. As for the meat of the matter, temporarily taking the authors at their word that they ONLY wish to advocate against student evals FOR THESE PANDEMIC TIMES, I believe that the matter is a close call.
On the one hand, student evaluations might reveal useful information about how much was perceived to have been learned on-line; how effective an instructor was in making the transition to remote teaching/learning;and whether individual classes were amenable to on-line instruction in the first place. In addition, information on some of the items listed by the authors — e.g., closed libraries, small group work, hands-on demonstrations, etc. — could be obtained that might be valuable if the virus continues or if universities begin to consider adding more remote offerings to their curriculum. (Incidentally, in many fields, there are streaming educational demonstrations available on YouTube and elsewhere for almost every possible topic. While no substitute for in-person learning activities, those videos are not that difficult to locate and do not take a lot of time to prepare. I’ve done it.)
On the other hand, results of a virus-tainted student questionnaire may not be fully valid because students may use it as a means to express frustration over the unexpected change to virtual education. They may not actually learn as much as they might have in a face-to-face classroom situation and their evaluations might reflect that basic belief. Similarly, instructors who do not adjust well to the changed circumstances may suffer unduly in annual reviews and T&P deliberations, especially if administrators judge this term’s student evals as they would a “normal” semester’s teaching record.
So, “close call…” If I were in charge, I’d continue the practice of student evals for this term but would…
1. Modify the questions to include several questions that solicited information and opinions on the on-line instruction;
2. Make sure that this semester’s results had a big ASTERISK next to them;
3. Make allowances for the abrupt and disappointing dislocations caused by the pandemic, including effects that are not directly related to course content and faculty performance — i.e., psychological considerations, changed living conditions, lack of on-campus employment, etc.
Faculty definitely need student evaluations more than ever to help them understand the problems with their online instruction and how to improve it. However, it’s not fair for evaluations in a crisis situation to be used for making tenure or hiring decisions, and colleges should be careful not to use these evaluations in an unfair way.
John K. Wilson said something similar to my posted comments — only he was MUCH more concise, diplomatic, and effective. Bravo, John K.!
A couple of quick (?) points. My university has announced that it will conduct course evaluations this semester, but that the results for this semester will not be used in promotion/tenure cases. This is perhaps consistent with a function of course evaluations that the authors of the essay do not mention: evaluations give students the impression (illusion?) that their opinions matter, and colleges and universities, in these neoliberal times, want to assure their customers that their views are important. In practical terms, of course, student evaluations that fall anywhere within a very broad range from poor to excellent are routinely ignored in promotion/tenure cases — at my R1 you can be a dreadful teacher and still thrive — and I imagine that it’s only in colleges in which teaching is the sole job a faculty member has, that the most wretched evaluations could be a source of administrative action. My solution, over the years, has been to invite students to make suggestions, air complaints, etc, on note cards that I pass out every week, collected by my TAs if I have them, left in my mailbox, etc: I get timely feedback on issues, and if there is an issue I can correct, the student(s) raising the issue can benefit, not merely gripe about it after the fact. Students have always said that they appreciate this, and if nothing else it assures them that I care about their satisfaction with me and with my courses.
Pingback: Equity and Human Rights during the COVID-19 pandemic | Concordia University of Edmonton