And this is good old Boston,
The home of the bean and the cod,
Where the Lowells talk only to Cabots,
And the Cabots talk only to God.
John Collins Bossidy’s old saw kept running through my mind as I read Thomas Frank’s “All These Effing Geniuses: Ezra Klein, Expert-Driven Journalism, and the Phony Washington Consensus” on Salon.com this morning. I also kept thinking of Paul Krugman’s continual complaint about economists who are constantly shown to be wrong, particularly when predicting inflation, yet who are continually turned to as experts. Oh, I didn’t forget John McCain, either, with his drum-beat for war, no matter the situation, wrong though he consistently turns out to have been–and be. The term “the echo chamber in Washington” has become cliche for a reason.
But mainly I thought about us academics.
Frank writes:
The powerful in Powertown love to take refuge in bewildering professional jargon. They routinely ignore or suppress challenging ideas, just as academics often ignore ideas that come from outside their professional in-group. Worst of all, Washingtonians seem to know nothing about the lives of people who aren’t part of the professional-managerial class.
There should be a big “ouch” there for all of us in the universities. We’ve become the go-to example of Lowells and Cabots.
And we’re doing little to change.
A big part of the problem is our slavish worship of established peer-reviewed journals and of the sanctified group of top-drawer university presses. There’s even a circularity going on: The more we strive to publish in these venues, the more valuable they become and the more expensive. The more expensive they become, the fewer who have access to them (outside of institutions, certainly), making their conversations directed to smaller and smaller cognoscenti. At the same time, those striving to climb the academic ladder have to conform to their demands or be consigned to the wasteland of non-R1 institutions or, even worse, to a life as contingent hires. Worse yet, those institutions trying to climb toward top status ape the requirements of the top universities, requiring that their faculties publish in venues where they have to compete for space with people with much lighter teaching loads, better support, and graduate-student assistance.
This also leads to the assumption that the faculty at Harvard, Yale, Stanford, M.I.T., Duke, Chicago and the rest are better than anyone else–and at everything. One of the more eye-rolling assumptions about Massive Open Online Courses (MOOCs) was that the “master teachers” at Harvard and Stanford could, through them, share their teaching genius with the masses.
The academic deck has always been stacked in favor of those who, for whatever reason, land at the elite universities. Not just students, but faculty. I admit that I have taken advantage of it: I got into grad school in part because of the reputation of my undergraduate college. And the status of my graduate school, particularly in my field, certainly hasn’t hurt me, since. But we in the universities, many of whom speak frequently of equality and commonality, do little to actually encourage these.
Certainly, we do little in terms of broadening scholarly conversations or encouraging alternatives to the hoary and narrow avenues of the past. We approach online open-access publication with suspicion when it is put forward at promotion and tenure time; self-publication has become an absolute no-no. Many of us talk the talk of openness but, when it comes to actual professional advancement, we won’t walk the walk. We even go so far as to put responsibility for this on the candidate: they have to prove that work not done in established venues is of value–relieving promotion and tenure committees from the necessity of reading and evaluating the work for themselves. If it has the imprimatur, fine. If not, don’t ask us to rate your work–show, instead, that others have already done so!
Academic procedures for promotion and tenure are, at most American institutions, arcane and out-dated, and open to abuses of all kinds. They also help keep our reputation as inbred professionals intact. We, as faculty, are the ones most responsible for it; it is we, as faculty, who can change it.
Will we?
Or will we continue to be supportive of a snobbishness as stultifying as that of old Boston–or even contemporary Washington, DC?
Another long-standing problem is the practice of not hiring students into tenure-track positions at the same institution from which they graduated, under the premise of: “*We* had to leave our university when we graduated and find a job elsewhere, so so should everyone else!” What kind of logic is that?
Thank you, Aaron, for this provocative and important post. But as much as I can agree that too often we faculty members demonstrate a hidebound and even stultifying in-group conservatism, there is also another tremendously important side to the story. Our most fundamental principles of academic freedom in teaching and research and of shared governance are based on the notion of professional competence and self-government. Peer review is certainly a system with warts, but it is the foundation on which our professional standing is based. Without it, who, might I ask, would make determinations of competence? Administrators? Trustees? Politicians? Surely those groups would likely to be even more inhospitable to iconoclastic views or innovative approaches than are the faculty.
I recall, for example, an incident early in my career when the dean at an institution where I was then teaching came to our department and opined that some study had revealed that the average historian publishes something like 3.7 articles every several years (I can’t recall the specifics) and that this should be the standard we use for promotion. Of course, in that same department there was a European historian who had published only one article in five years, but it had won a prestigious award and helped its author obtain a Guggenheim fellowship. By contrast, another member of the department, who did local history, was churning out numerous puff pieces for the local historical society. The dean’s presentation helped explain why the latter was an administration darling while the prize-winning (but sometimes ornery) Europeanist was merely tolerated.
In short, while your critique may be spot-on in many, many cases, I’m not so sure I can agree that we faculty are the ones most responsible for the flaws in our systems of peer review. In my career I have seen instances in which tenure or promotion decisions were unfair, biased, or simply stupid. But in most cases the culprits were not hidebound faculty colleagues, but cynical and authoritarian administrators. And in the great majority of cases that I have seen, the system, for all its flaws, works reasonably well. Moreover, what you dismiss as “arcane and out-dated” procedures are often, especially when sanctified by collective bargaining agreements, an important means for defending both professional quality and the due process rights of individual faculty members.
Dirty linen does need to be aired, that’s for sure, but let’s not forget that there are those out there for whom it is not the dirt but the linen that they wish to expunge. All professions must contend with the tendency to become inbred and to “circle the wagons” in the face of criticism, medicine and law as much as academia. But professional autonomy, especially in academia, has powerful enemies for whom our flaws are not problems to be corrected but excuses for advancing an insidious agenda.
Nonetheless, my thanks again for initiating this conversation. And one final point: I wish you had written SUPPOSED “wasteland of non-R1 institutions.” For many of us teaching at non-research comprehensive colleges and universities, this is where we WANT to be. Far from surviving in a wasteland, we live and often thrive in the still-beating heart of American higher education.
Thanks… and I will respond more later. And “wasteland” is certainly meant as sarcasm on elite attitudes… so “supposed” is right.
I’m not against peer review, but against using peer review as an excuse and a deflection. Sometimes I get the sense that people use the fact that something has been peer reviewed as reason to not bother to look at it themselves. It is also used as a means of escaping decision-making oneself. “Oh, it is peer reviewed…. It must be good.” What bothers me even more are the abuses of blind peer review, where someone can use anonymity to further a personal agenda or vendetta.
My real beef, however, is with a system that now has so many layers and so many levels that the process of tenure and promotion begins to seem more like a maze with constantly moving paths rather than a route that one can reasonable follow. I see so many junior faculty so unsure of what the expectations of them are… to the extent that I have actually seen people shaking, so perilous do they see the shape of their careers. This should never be the case. Certainly, it is not necessary.
I think we, and our bargaining units, can work to devise a revision to the system that will still protect professional independence while ridding it of the worst of the abuses that do occur.
Aaron writes: “I see so many junior faculty so unsure of what the expectations of them are… to the extent that I have actually seen people shaking, so perilous do they see the shape of their careers. This should never be the case.” I couldn’t agree more; indeed, what decent person wouldn’t agree. But, alas, I fear this may be a product less (or at least not only) of confusing policies and procedures than of a pervasive academic culture that will be difficult to change. At my campus we have a fine faculty development office that works with probationary faculty from day one, a strong union contract and a relatively effective system of shared governance to protect individual rights, and a longstanding and detailed tenure and promotion policy with (mostly) clear criteria designed by faculty. Probationary faculty also undergo annual reviews each year before coming up for tenure so that problems can be identified and corrected early. Most important, of those who come up for tenure over 90% get it, although we do enforce rigorous standards, especially with respect to teaching. Yet I too have seen people so anxious about the process as to endanger their health, including a few who are known to everyone as highly qualified “slam dunk” cases. We try to counter this with informal mentoring, but the situation persists, worse in some departments than others, but still fairly extensive. I fear that because tenured positions have become rarer and rarer, the very concepts of “tenure” and “the tenure decision” are taking on a mystical aura that can become frightening in and of itself, policies and procedures notwithstanding.
Interesting points, Hank. You are certainly right that the increased rarity of tenure is making junior faculty lives more difficult. What I am seeing, also, is that changing professional standards, many times resulting from the growth of digital possibilities, are also keeping people on edge. Junior faculty are often told to be progressive in their attempts at publication but, later, find out that they should have stuck to more conservative venues. Few people seem to be able to say with certainty what will “count” and what will not. More and more, I am leaning toward outsider reviewers, especially since people in many departments are now judging people by standards they were not judged by and that they don’t really understand. I do not mean to put those senior faculty down, but simply to suggest that we recognize that things are changing more rapidly than any individual can generally keep up with and that we should rely on the advice of others when the changes are not ones we are directly involved with in our own work.
This is a fascinating discussion and I hope our audience is not shrinking too much. I agree that outside referees can be quite helpful in assessing scholarship, although where there are real differences over what constitutes “good” work in a field the conflicts might simply be displaced into battles over which outsiders should be queried. At my campus we don’t use outside evaluators, but informally we frequently encourage candidates to solicit letters from those familiar with their work, especially when few on our own faculty are really in a position to appreciate what they do.
More problematic is the assessment of teaching, in which outsiders can offer little help. We say that teaching is primary, with research and service secondary, and I suspect that is true in many, if not most, primarily teaching institutions. But often we are at a loss about how to assess instruction. Some — mainly, but not only — administrators, look almost solely at quantitative measures, most often student evaluations. Others seek to measure “quality” and yet others “innovation,” both terms being exceedingly difficult to define in ways that satisfy everyone. For (too) many of us good teaching is like obscenity: hard to define but we know it when we see it. No doubt this “squishy” quality contributes a great deal to the anxieties and uncertainties that we bemoan.
In my senior year at Boston University, Howard Zinn said in a course on civil liberties and civil rights: Maybe the Lowells talk only to Cabots, And the Cabots talk only to God. But the workers can interrupt the conversation!
Hank and Aaron are both right: peer review is a core necessity of academia. But that ideal is betrayed when elitism prevails. In order to have peer review, you must have the “peer” part (faculty control of hiring) and the “review” part (actual analysis of quality). When faculty use shortcuts (solely looking at the quantity of work, prestige of Ph.D. program, prestige of presses or journals, prestige of recommenders, size or source of grants) instead of reviewing a candidate’s actual work, they are betraying the “review” duty of “peer review.”
Ironically, one of the core principles of “peer reviewed” journals and presses is that they’re supposed to utilize blind review—to judge the merit of the work without regard to the reputation of the researcher or the institution where it was done. That’s precisely the attitude that should prevail in hiring and admissions, and rarely does.
I think that a major part of what Aaron is critiquing in the current system boils down to the increasing corporatization of scholarly journals–of not just their production and dissemination, but also, still largely indirectly but nonetheless increasingly, of their editorial management. This trend poses at least as much a threat to faculty control of the “peer review” process as looking for alternatives to the current system might do so.
In another post, Aaron described the alternative of ongoing peer review of scholarship disseminated in online forms or forums, whether those might be digital journals or digital collections on special topics or something else entirely. Undoubtedly, there would be many problems inherent to such an alternative system that would have to be addressed. But scholarly publishing seems paradoxically to be proliferating and to be pouring toward the narrow end of a funnel. Having some alternatives in development would seem a prudent course to pursue. And open-source scholarly publishing will certainly eliminate the corporate middlemen and return the process more fully to the control of the faculty themselves.
Every tenure-track hire or promotion that I have participated in, includes very thorough review of the candidates papers by several people judged to be experts. I’ve been in that position myself, and I don’t count pages or impact factors, I explain the significance of the publications for the field. This is the only standard I have ever seen used.
That’s wonderful, Steve. And it is all the more reason to be concerned about the people who take shortcuts or use the process for inappropriate ends. They make all of the rest of us look bad… and it only takes one person on a panel to sidetrack the work of the whole.
Also, I believe that the use of outside experts, especially in a climate of uncertainty, is a good idea. I wish it were used more frequently.
Our R1 institution has about an 80% rate for assistants getting tenure (or so I’ve been told—I’ve only seen two faculty members actually denied tenure in 28 years, and both were pretty clear cases who had been warned about their low chances 3 years earlier).
Although there are good open-access journals, a lot of the open-access journals are just rip-off publishers interested in taking money from authors to up their publication counts with no attempt at quality control. Publishing in those journals is evidence that the author either knew that the work was junk or did not do their homework to find out what sort of journal they were publishing in. Although the “prestige” journals have acquired too much weight (both Nature and Science have high retraction rates for publishing junk science), the proliferation of junk journals is not helping things.
Our department has very diverse researchers with such a broad range of different subjects that most of us cannot fairly judge the work of our colleagues—we rely very heavily on outside reviews for the 3-year review, the promotion to tenure, and the promotion to full professor. Publication in carefully reviewed journals is important in the process, as a check that people in the subfield of the researcher, who can more fairly judge the work than anyone in the department, have looked at the work and found it reasonable.
The problem, gasstation, is assumptions that may or may not be warranted about work based on its venue of publication. Publication in certain journals is NOT “evidence” of anything. People choose venues for all sorts of different reasons–including belief that the echo-chambers of their fields will not consider their “original” work (whether this is true or not is a different story). Also, be careful of equating “online” with “junk.” The two words are not synonyms.
As to judgment, we have developed (and continue to develop) open review processes that much more powerfully assess work than does blind review prior to publication.
I do not say you are wrong in what you claim, merely that your position is incomplete in a fast-changing scholarly environment… which is exactly the problem.
I don’t equate “online” with “junk”. In fact, in my field the online journal PLoS Computational Biology is highly respected. But I get 1-2 spam messages a day from junk journals, almost all of which are online—there are a lot of junk publishers online, because it costs almost nothing to start an online journal if you don’t care about quality.
I have considered publishing in some online journals (but since I was without funding for 5 years, I was not able to afford the author fees). I might still publish something in PeerJ, which has a more reasonable pricing model than most of the online journals.
A lot of the open review processes that people have experimented with have resulted in no review for a lot of poor papers, as no one thought the papers worth the trouble of commenting on. It is hard to separate good papers that have not attracted attention from bad papers that have not attracted attention in an open review process.
Prior peer review saves readers the time of picking through a huge slush pile, at the cost of losing a few good papers. Most of us don’t have the time for reading even the pre-filtered stuff, much less picking through the stuff that wouldn’t have passed peer review.
I agree that publishing in a “prestige” journal is no guarantee of quality. Nature and Science particularly seem to select papers more for how sensational they are than how correct or careful their science is.