No matter the metrics devised, grading is subjective.

All grading.

How can I say this?

Don’t the scales created offer objective data?

No. The decision-making in creation of the scales is necessarily subjective itself, making all evaluation using the scales just as subjective. Yet we continue to believe in the objective status of grades. Here in New York City, we force all restaurants to display a grade from the Health Department–and we all accept that those establishments marked “A” are cleaner and healthier than those market “B.” But do we know why one is given an “A” and another a “B”? Not often.

When I ran a cafe, back in the days before this system, I would be faulted for not having an automatic door-closer on the restroom (because if its location, that wasn’t needed… but rules are rules)… and my internal waste line did not quite meet specs (which didn’t matter, for the main waste line of the building was twice the size required, making the back-ups worried about next to impossible). I got high marks for the restroom itself, for it could easily accommodate a wheelchair, even though few wheelchairs could get through our front door easily–which could not be changed because of the landmark status of the building. Would I, today, have gotten and “A” or a “B”? I don’t know. Either way, my coffee drinks would have been the same, as would have been the pastries and other edibles, none of which was cooked on site.

Also, different inspectors emphasized different things, even though they were looking for the same types of problems, working from the same lists. One would skip something as trivial that another would write us up for.

Translating all of this to a letter grade does not make restaurants cleaner or healthier. It simply makes us feel we know something at a glance in the window.

We do have a need for grading. In our schools, we have developed a system that accounts for the vagaries of individual teachers and styles and even of subjects. That is, students are graded for each course and (generally) by a different person with somewhat different standards in each instance. We then use an aggregate (the GPA) to evaluate the student’s progress. This makes problems of bias and attitude less likely to skew the result tremendously. When someone comes out consistently near the top, no matter the grading scale and no matter the person doing the actual evaluating, we can safely say that this person is performing above someone consistently near the bottom. The grade from the Health Department is based on no such aggregate of differing standards and inspectors. It is based on a supposed single standard and single inspector. There’s no GPA, no indication that the institution has earned “A”s consistently in the past or that the “B” of today may be an anomaly. The lack of uniformity of our grading system, in other words, is one of its strengths, for it allows us to use a system accounting for the subjective differences in initial criteria and for the subjective differences in use of those criteria. It is far from a perfect system, but it does have a built-in corrective for the worst abuses.

In New York City, today, we are also presented with letter grades for our schools. These have all the disadvantages of the grades on restaurants. Schools are being closed because they do not meet what are, in many eyes, arbitrary and capricious standards set by a single reviewer. The corrective of multiple types of reviews by multiple reviewers does not exist.

Today, I read in the New York Times, that Michelle Rhee’s group StudentsFirst is grading entire states on their systems of education. Coming in highest are Florida and Louisiana, with New York and New Jersey scraping by with “D”s:

Ms. Rhee said that the relatively weak showing reflected how recently statehouses had begun to address issues like tenure and performance evaluations. “We didn’t say in any way that we want to show people how bad it is,” she said in a telephone interview. “We wanted to show the progress that is being made, but in places where progress is slower to come, be very clear with leaders of that state what they could do to push the agenda forward and create a better environment in which educators, parents and kids can operate.”

But it is Rhee who is defining “progress,” no one else. She has created the scale and then has done the evaluations. She reminds me of “cunning old Fury” in Lewis Carroll’s “The Mouse’s Tale”: “I’ll be the judge, I’ll be the jury… I’ll try the whole cause, and condemn you to death.” What Rhee is providing isn’t a system providing valuable information but is simply a means of making her own subjective sense of “progress” look objective.

Our mania for objective and standardized evaluation has led us to accept almost anything set forward as a “grade” as a thing of value. This is nonsense, and it has been leading to the destruction (or attempted destruction) of the American system of K-12 education. An to a building threat to American higher education.

And who is Rhee to be doing the grading, anyway? Generally, we like to rely on experts with proven track records.

What is hers?

She ran the Washington, DC schools for a few years. Are they better as a result?

Taken in the aggregate, both as schools and through differing evaluations and standards, they are not.

Shouldn’t she be scoring high on evaluations before starting to evaluate others?