BY HANK REICHMAN

Last week three scholars published an article reporting on how they had submitted over twenty fraudulent and purportedly ridiculous nonsense papers to various journals in gender and diversity studies.  Seven were accepted, four were published online, and three were in process when the authors “had to take the project public prematurely and thus stop the study, before it could be properly concluded.”  Two of the three, James A. Lindsay and Peter Boghossian, had previously boasted of placing a similar bogus piece, on the “conceptual penis,” only to be embarrassed by the disclosure that the journal they had hoaxed was essentially a pay-for-play scam.  (The third hoaxster, Helen Pluckrose, is a self-described “exile from the humanities” who studies medieval religious writings about women.)

The hoax was quickly dubbed “Sokal squared,” after the 1996 publication of a bogus paper by New York University physicist Alan Sokal.  It received extensive coverage in the Wall Street Journal, the Chronicle of Higher Education, and the New York Times.  The usual suspects, including Steven Pinker, who had prematurely applauded the “conceptual penis” fiasco, were gleeful.  Yascha Mounk, an author and lecturer on government at Harvard, wrote that the fraud “showcases a serious problem with big parts of academia.”  A more sober assessment came from McGill University libertarian political theorist Jacob Levy, who told the Chronicle, “I am so utterly unimpressed by the fact that an enterprise that relies on a widespread presumption of not-fraud can be fooled some of the time by three people with Ph.D.s who spend 10 months deliberately trying to defraud it.”

It had been my intent to comment on this scandal at length, but I was busy over the weekend (it was Hardly, Strictly Bluegrass week, after all!) and by now others have debunked the story far more completely and probably much more competently than I would have.  So instead I’m just going to refer readers to some links and slip in a few random remarks of my own along the way.

A good place to begin is with Daniel Drezner’s piece in the Washington Post.  Drezner, a professor of international politics at Tufts University, argues that the fraudsters’ own paper itself “would have never passed the muster of peer review.”  To be sure, he argues, “the authors got several papers accepted and favorable referee reports in fields like gender studies.  What is entirely unclear is whether other disciplines are equally vulnerable.”  He also finds that “the authors contradict themselves on several fronts.”

Several observers have agreed that other fields may also not be without similar problems.  Indeed, more often than not the problem is not with the acceptance of intentional hoaxes but with the publication of seriously submitted drivel.  Economics is a prime example.  Matthew Yglesias tweeted that “the ‘natural rate of unemployment’ is a clever Sokal-style hoax.”  Economist Noah Smith agreed, adding on Twitter that “Real Business Cycle theory is kind of a Sokal Hoax that won a Nobel prize. The idea was that recessions happen because humanity suddenly forgets technologies, causing workers to take a voluntary vacation in which they pretend to be ‘unemployed’. And it won a Nobel.”  In addition, the scope of the current hoax is dwarfed by the acceptance and subsequent embarrassed withdrawal of more than 120 computer-generated “gibberish” papers in the physical sciences that  made it into more than 30 published conference proceedings between 2008 and 2013, as reported in Nature.

Justin Weinberg, at Daily Nous, actually read two of the papers, unlike those who took the hoaxsters’ contentions at face value.  Here is what he wrote about one of them:

I read the article that Hypatia accepted, “When the Joke Is on You: a Feminist Perspective on how Positionality Influences Satire.”  In my opinion, if the citations are legitimate and the descriptions of others’ views are accurate (something which I am not in a position to determine at this time), the editors of Hypatia have nothing to be particularly ashamed of.  Most of the twenty-page paper is a reasonable synthesis of others’ ideas about oppression and humor.  It may not be groundbreaking (as one of the reviewers points out), but it is not ridiculous.  It seems to me that only on the last page of the paper are there certain statements that could be interpreted as outrageous, but they are so vague that a much more charitable alternative interpretation would be reasonable.  In short, assuming accurate representations of others’ views and legitimate citations, one’s opinion of Hypatia should not be affected by its publication of this paper.

“What does this hoax show, if anything?” Weinberg asked.  “For one thing, it shows that academic publishing is not particularly adept at engaging with those who are operating in bad faith and intending to fool the system.  It also shows that a system which is set up to assess scholarship critically but charitably will have false positives.”  But that is not what the hoaxsters wanted to show.

Another interesting take came from blogger Michael Keenan, who judiciously contends that the “hoax is over-hyped and the reporting on it is terrible.”  Keenan includes links to all the papers and the reviewers’ comments.  About the latter, he sympathetically concludes, “After reading the reviewer comments, I’m very sympathetic to the reviewers, and I update toward thinking that their field is not a made-up illegible jargon-fest. . . .  I’m sad to contemplate the reviewers trying to help someone fix the mistakes in their paper, while the authors’ intention is to slip through as many mistakes as possible.”

The hoaxsters claimed that one of the fields “exposed” by their effort is Sociology, although none of seven sociology journals to which the pieces were submitted accepted them.  But a reviewer for one such journal, a doctoral candidate at UCLA, documented his response to the hoax on Twitter.  Here is what he wrote:

I was Reviewer 1 for the Masturbation = Rape hoax paper that tried to get published in Sociological Theory. As a grad student, it was my first time being asked to review a paper for a journal.  I’m glad I recommended a reject, and the paper was rejected.

I remember thinking at the time that it was probably a master’s thesis that a student immediately turned around to try to get published.  Lots of long block quotes with no explanation.  Long sections with no organization.  I mentioned this all in the review.

So I structured my review off of a constructive rejection I received from ASQ where the reviewer clearly read the paper, pointed out problems, and offered suggestions for how to proceed.  It was the type of rejection where I immediately wanted to work on the paper again.

I don’t like reviews that reject the premise of the paper outright.  I’ve received reviews like that since my papers are on the porn industry.  So I tried to buy into the paper and offer paths forward.  These are the comments that the hoax authors quoted in their write up.

Anyways, I guess I could be more critical in the future, but I assumed a grad student had written a confusing paper and I tried to be constructive.  I’m embarrassed that I took it as seriously as I did, I’m annoyed I wasted time writing a review, and I’m glad I rejected it.

If this poignant confession reveals anything it is that there is much that is actually right with the peer review system.  Weak essays are published all the time, of course.  Many should be, because it is only through the open confrontation of ideas that knowledge progresses.  I’m reminded of a time quite a few years ago when a colleague came to my office door embarrassed that he had earlier recommended rejection of a piece that had now been published elsewhere.  With that publication he discovered that it had been written by one of the towering figures in his field!  “I thought it was a nice first effort by a grad student,” he sheepishly confessed.  Yet maybe he was right to reject it.  And it may also have been right for another journal to publish it.

One of the hoaxsters’ most ardent defenders online has been Mounk, who concluded a lengthy Twitter thread with what he called “the crucial point: Though most parts of academia retain serious intellectual standards, Sokal Squared suggests that you can now be made a professor, and get to teach college kids, by spewing absurd, ideological bullshit.”

In fact, it is this claim that is bullshit.  You don’t become a professor by spewing ideological absurdities in hoaxed papers.  You become one by completing a rigorous course of study, writing a dissertation assessed by experts in your field, and convincing similarly constituted hiring, retention, promotion and tenure committees of your professional worth.  To be sure, none of that is a tried and true prophylactic against the possibility that one day you will produce work that is bullshit, but such bullshit is not now and never has been the road to becoming a professor.

The authors of the paper claim that they have exposed the hypocrisy and shoddiness of fields they flippantly label “grievance studies.”  But as Drezner concludes,

Speaking of grievance, the study’s authors seem to possess ample amounts of it.  According to the Wall Street Journal’s Jillian Kay Melchior: “Mr. Boghossian doesn’t have tenure and expects the university will fire or otherwise punish him.  [I hope they don’t, unless he wants to argue that this escapade represents the totality of his scholarly work.-HR]  Ms. Pluckrose predicts she’ll have a hard time getting accepted to a doctoral program.  Mr. Lindsay said he expects to become ‘an academic pariah,’ barred from professorships or publications.  [Judging from the plaudits they’ve received, however, these fears seem overly fraught.-HR]”

I hope the authors get past their own emotions about these disciplines to take a more stringent look at their strengths and weaknesses.  Their paper does point out some weaknesses, to be sure.  But the brush they used to paint their portrait of the humanities is way too broad.

Or, as a post on Daily Kos concluded, the hoaxsters “should probably have put more time actually critiquing constructivism rather than punking its practitioners.  There are thought experiments and thoughtless experiments. . . . Submitting absurd academic papers, while amusing, is pretty much still the equivalent of Trump tweets, always dangerous if assumed to be true.  It’s just not that ‘giant’ a hoax, as much as it’s taken on some of the proliferation of less-rigorous research and even more subaltern fields.”