The blog of Academe Magazine. Opinions published here do not necessarily represent the policies of the AAUP.
Yesterday, I posted on the ‘digital badges’ that some think may point the way toward an alternative form of educational certification. I don’t think that’s a good idea, but I didn’t really explain why (saying I would later get to why that’s not the same as a college degree). This morning, I saw a link on Diane Ravitch’s blog to an article by Esther Quintero, a sociologist who writes for the Albert Shanker Institute’s Shanker Blog. Entitled “The Data-Driven Education Movement,” it explains why data alone is insufficient for educational analysis.
Quintero points out what should be obvious but is not, that we rely to readily on data that can be quantified easily thus limited the data universe we consider, and that we assume incorrectly that there’s truth in data in and of itself. Thing is, as Quintero says:
(1) numbers are not the only type of data that matter; and (2) all data need to be interpreted before they can be elevated to the status of evidence – and theory should drive this process, not data.
I would add something else that Quintero talks about, (3) the fact that theory should drive the process is also a limitation–so theory needs to be constantly questioned as data is accumulated and assessed.
She uses as an illustration of the problem of our use of a limited data universe a story I know somewhat differently–but the point is the same. Someone who has lost something is seeking it under a streetlamp. When asked why there only, the person says, scornfully, that it is night, so they couldn’t see anything elsewhere… so why look where you can’t see?
Quintero then talks about how we often use data to develop theory rather than using it to test theory. The problem with this is that theory developed from a limited data set (and all are limited) is always going to be incomplete–even if it is consistent. Data, in other words, is better used for proving theory wrong than in generating theory out of whole cloth. Yes, it can be used as an impetus for modifying or rejecting theory, but no set of data alone should be the force behind creating theory. It’s just too limiting. Kurt Godel, in his famous proof, showed (to put it most simplistically) that no system can be both complete and consistent. If it is complete, it contains contradiction. If it is consistent, it does not cover all possibilities. What we have with theory that is solely data driven is movement toward consistency–but that, again, only accounts for what we can see under the streetlamp.
Quintero ends with this:
Our blind faith in numbers has ultimately caused impoverishment in how (and what) information is used to help address real world problems. We now apparently believe that numbers are not just necessary, but sufficient, for making research-based decisions.
The irony, of course, is that this notion is actually contrary to the scientific process. Being data-driven is only useful if you have a strong theory by which to navigate; anything else can leave you heading blindly toward a cliff.
Real education is a great deal more than reaching numerical benchmarks (though they do have their place). ‘Critical thinking,’ though a tired old phrase bowed to and ignored (for the most part) today–and not simply skills–is at its core. As ‘critical thinking’ deals with those things outside of the glare of the streetlamp, our data analyses can’t encompass it, so it gets passed by in favor of things that can be counted, things we now bow to as information that can be assessed repeatedly with the same conclusion reached each time.
That’s inherently regressive, always looking back, wanting to repeat what has been done before. The rubrics for writing exams, no matter how flexible we want to make them, always seem to drag the writers back to the five-paragraph theme. Why? Because they always focus on things that can easily be counted (paragraphs, sentences, spelling errors, sentence types, punctuation variety, supporting points, etc.). They have to, or the conclusions reached will be different for different readers.
The problem with that? Someone who writes brilliantly, but in another fashion, will always fail–even when the reader knows the writing is better than anything encompassed in the grading rubric. The reader isn’t given the freedom of deciding on his or her own, but only to make a decision based on the ‘countables’ of the rubric.
It is this freedom, one that allows for a real interaction between teacher and student, that is at the core of real American education–and is why our education is so good. And it is this freedom that digital badges (which only deal with what can be found under the streetlamp) do not provide.
And that, of course, is why digital badges will never be a real challenge to traditional structures of education–as long as those overseeing those structures don’t cave in to the contemporary mania for simplistic data-driven assessment.