This is a guest post by Darren L. Linvill, an assistant professor in the Department of Communication Studies at Clemson University. His article, “The Bias Fallacy,” appears in the newest issue of Academe.
Did you know people who like mayonnaise are more likely to be good dancers? As my undergraduate research methods students are taught, correlation does not equal causation. This, and other, foundational concepts of sound methodological practice are not always adhered to by some researchers working on behalf of the National Association of Scholars.
I haven’t examined the full breadth of research and commentary published by the NAS. I’m confident that if I did I would find some good work coming from competent and dedicated scholars. If I wanted to make an argument that the NAS publishes poor scholarship, however, and sampled only the NAS’s two most recent reports, A Crisis of Competence and Recasting History, I could make a convincing case. This same kind of cherry-picking is only one tactic employed by these reports that invalidates their broader claims. Selective sampling may not be sound research practice, but it certainly makes a researcher’s job easier. It is not difficult to argue a point when you only examine the data that supports that point.
The NAS’s two recent reports are both broadly aimed at addressing what that organization, among other critics of academia, view as an ongoing politicization of higher education. In my recent Academe article, “The Bias Fallacy,” I discuss the evidence put forth by the NAS in A Crisis of Competence: The Corrupting Effect of Political Activism in the University of California. In the report NAS attempts to link university politicization in the University of California system to recent declines in basic skills among graduates. The report attempts to illustrate a liberal bias in the University of California system predominately through clearly cherry picked evidence such as student narratives and course descriptions. While the research methods employed are biased, this is not where the report truly falls short. Its greatest weakness is in failing to establish a causal link between supposed classroom politicization and any negative student outcomes.
Society and higher education have both seen an incredible variety of changes in recent decades. Any number of factors may have led to an impact on certain student skills which A Crisis of Competence refers to. While college professors are predominately politically liberal, it is possible to be liberal while remaining a competent educator who teaches from multiple perspectives and evaluates students’ ideas based on factors other than ideology. The perception of political bias in the classroom should be addressed, but not for the reasons the NAS suggests. In my article I discuss what peer reviewed research tells us about the role of ideology in the classroom and just how I think the issue should be approached.
In publishing fallacious reports such as their recent work, I can’t help but wonder if the goal of the NAS is less to engage in scholarly discussion and more to persuade pliable minds, just as they accuse liberal professors of doing.
You said (discussing NAS study on American History in Texas)
“this same kind of cherry-picking is only one tactic employed by these reports that invalidates their broader claims. Selective sampling may not be sound research practice, but it certainly makes a researcher’s job easier. It is not difficult to argue a point when you only examine the data that supports that point.”
This criticism is not valid–since there was no cherry picking or selective sampling done in Texas Study. Every single reading assignment assigned by every faculty member was read completely and classified into 11 possible areas of historical themes. Most were classified as having more than one theme, if the assignmemt merited it.
The person who did the classification was a “blind reviewer” and did not know how the readings assignment codings he did would be used.
The Study showed differences between UT and A&M.
So actually, standard social science design was followed.
Richard Fonte, researcher for NAS Texas Study
Regardless of how the data set was analyzed, in choosing only one form of data the NAS Texas Study engaged in cherry picking. The study may have made some valid claims, but it also made several broad assertions that were over-reaching given that it only looked at one form of data. To say that students in Texas are getting a less than comprehensive view of history is a very big claim and I question if other forms of data would support it. A great deal happens in the classroom beyond course readings.