BY JOHN K. WILSON

This week the College Free Speech Survey was released, which was developed by the Foundation for Individual Rights in Education (FIRE), the conservative website RealClearEducation, and the research firm College Pulse, and funded by the Charles Koch Institute. It is being marketed as the largest survey ever conducted of students about free speech, questioning 19,969 undergraduates at 55 colleges. While a few aspects of the survey may be useful and noteworthy, the rankings of colleges derived from it have numerous and serious flaws that render it useless as an assessment of free speech on individual campuses.

The report (pdf) declares, “The College Free Speech Rankings are the first tool to provide such detailed, quantitative insight into the relative environment for open dialogue at colleges under consideration, helping prospective students and their parents as they assess different colleges.”

In reality, these rankings are worthless, the quantitative insights are incoherent, and prospective students and parents should simply ignore these rankings.

Remarkably, 40% of the college rankings were determined by the student responses to this set of questions:

“Would you support or oppose your school ALLOWING a speaker on campus who promotes the following idea:”

• “Abortion should be completely illegal?”

• “Black Lives Matter is a hate group?”

• “Censoring the news media is necessary?”

• “Some racial groups are less intelligent than others?”

• “The U.S. should support Israeli military policy?”

• “Transgender people have a mental disorder?”

What do these six statements have in common? They are things conservatives say. What, you couldn’t come up with even one question that might offend a conservative, let alone an equal number? In reality, the survey did ask two questions about speakers who might offend conservatives, including “All white people are racist” (banning this speaker was heavily supported) and “Christianity has a negative influence on society.” Yet, shockingly and inexplicably, these two responses were simply removed from the results for the rankings of the colleges on free speech.

Can anyone take seriously a ranking where a key question asks about banning six conservative speakers and only two liberal speakers, and then drops the results about the two liberal speakers?

This report reveals that the average overall college score from a liberal student was 49.79, while the average conservative student score was 53.14, which according to the report means that conservatives say there is “a better climate for free speech and expression” on campus than liberals. Some people might point to this (incorrectly) as proof that conservatives think there’s more free speech on campus than liberals do. In reality, it simply means that this one set of key questions about support for censoring speakers only asked about banning conservatives, and conservatives were naturally less likely to support that, resulting in a higher score. For example, on this tolerance factor at the University of Chicago, conservatives had a rating of 71.4 and liberals had a rating of 52.6. As a result of this single distorting factor, colleges that enroll a lot of conservatives (including public colleges in the south and religious colleges) jumped up in the rankings. I cannot imagine why the data was manipulated in this bizarre way.

It is disturbing that so many students support censorship, but it’s hard to know how many students misunderstood the question as an expression of opposition to these offensive views. Even if it was correctly done, this question would be terrible to use as 40% of the score for ranking colleges because it says nothing about the college’s policies on free speech or what it does to educate students; instead, a college is blamed simply if it admits students who support censorship.

The rest of the determination of the rankings are also deeply flawed. Another massive 40% of the ranking is based on the answer to whether it is “difficult to have an open and honest conversation about on your campus” on eight topics. The problem here is the word “difficult.” Some campuses work hard to encourage students to have “difficult conversations” and when the word is mixed up with censorship, it could confuse some students. It is in fact difficult to have an open and honest conversation most of the time about controversial topics. Colleges are punished in the rankings if they encourage difficult conversations because students will be more likely to report feeling uncomfortable about these conflicts. But the important thing is whether the conversations happen and are encouraged, not whether they’re difficult. The topic in response to this question with the highest “difficult” response was abortion, while the lowest was feminism. If students were actually terrified of campus “feminazis,” you wouldn’t have this large gap between two related topics. This indicates that some students may consider conversations “difficult” if they think the topic itself raises difficult dilemmas (as most Americans do about abortion), not because people can’t speak about it on campus.

And 12% of the total score in the rankings comes from this question: “Personally ever felt you could not express your opinion on a subject because of how students, a professor, or the administration would respond?” There are many problems with this question, which 60% of students answered yes to. First, it is fully subjective. If a student is easily intimidated, they could feel unwilling to express themselves simply because someone might argue with them. Even if a college promotes free expression and controversy, a single bad experience would cause a terrible score. By contrast, a college where people party and rarely debate ideas would get a high score because no one feels reluctant to speak if you’re never challenged. This question is also troubling because it merges punishment for speech (being suspended by the administration or failed by a professor) with feeling bad about a reaction. Almost all of the positive responses to this question probably came from fear of criticism from fellow students. This question also raises feelings that are almost universal and have nothing to do with free speech at a college. I suspect that most people outside of college would say that they have felt once during the last few years that they could not fully express their opinion because of how friends, co-workers, bosses, and online trolls might respond. As a writer, I constantly censor myself by editing out arguments based on how my readers will respond. Pretty much the only people who never censor themselves due to the reactions of others are the ones who constantly censor themselves by not discussing controversial ideas.

The survey found that conservatives report higher levels of self-censorship than liberals (72% vs. 55%), and African-Americans self-censor slightly more than whites (63% vs. 60%). But if students mostly report self-censorship because they don’t like facing criticism, then you would expect people who form a minority on campus (such as conservatives, or African-Americans) to report higher levels of self-censorship, even if no one is actually suppressing their speech.

The speech code rating from FIRE is worth 4% of the total score (although, oddly, the ranking subtracts four percentage points for a Red Light rating, which means it’s actually a much larger factor).

Sadly, the two most important and well-written questions in the survey are worth almost nothing in the total score for the rankings: “Does your college administration make it clear to students that free speech is protected on your campus?” and “If a controversy over offensive speech were to occur on your campus, would the administration be more likely to Punish the speaker for making the statement or Defend the speaker’s right to express their views?” These are useful questions (albeit vague and speculative), but they are only worth 2% of the total score each in the rankings.

Other questions in the survey were completely ignored in the rankings. For example, the University of Chicago was ranked #1, but it apparently had among the highest proportion of students who thought it was acceptable to tear down flyers you don’t like.

Unfortunately, the deeply flawed rankings draw attention away from some of the important questions raised by the survey. For example, “More than 60% of extreme liberals said it’s “always” or “sometimes” acceptable to shout down a speaker; compared to 15% for extreme conservatives.” This is an extremely disturbing view, since shouting down speakers is never acceptable. It should be noted that those on the left prefer to censor speakers by shouting them down, while conservatives prefer to censor speakers behind-the-scenes, so we shouldn’t be surprised at a large gap in support for a predominantly leftist tactic. It doesn’t necessarily mean that conservatives censor speakers less, but it does indicate a basic desire for censorship that colleges must challenge.

This might be the largest study of students on free speech ever conducted. But in survey research, bigger is often not better. A bad survey of 20,000 students is a lot worse than a good survey of 500. And this is a bad survey because a lot of the questions are flawed and the rankings drawn from them are deeply distorted..

As bad as the rankings themselves are, some of the media conclusions have been ever worse. A ridiculous headline on RealClearPolitics (part of the same company as one of the report’s sponsors) announced, “College No Place for Free Speech Fans, Rankings Show.” On RealClearEducation itself, the headline was, “2020 College Free Speech Rankings Reveal Crisis on Campus.” The article claimed, “do college students really feel free to speak their minds on campus? Newly released College Free Speech Rankings show that, at most colleges, the answer is no.” This is false. In reality, the survey never asked students if they feel free to speak their minds (probably because they would mostly say yes, and why ruin good propaganda with the truth?). There is a huge difference between feeling free to speak in general and recalling one time during your life at college when you were reluctant to speak your mind.

FIRE Executive Director Robert Shibley declared, “These rankings provide proof that your choice of college can make a real difference in your ability to speak your mind,” No, they don’t. There is no “proof” of anything in the rankings. The real difference in your ability to speak your mind is you. If you decide to speak, you can speak at almost every college in America. What we need are colleges that reject policies of censorship and help encourage the discussion of different ideas on campus. But these rankings, far from being proof of that, are essentially worthless at determining which colleges are doing that. Creating a culture of freedom and openness on campus is a difficult thing to do, it’s a difficult thing to survey, and it’s a difficult thing to rank, but these rankings completely fail to help us identify which colleges do that.