When I teach my technical-writing students about executive summaries, I tell them to imagine that their boss is either too dumb or too hurried to look carefully at the material behind the summary. They laugh, but they get the point: the boss (who is probably smart, actually, and a good judge of time) doesn’t want to be bothered with the details of a report unless she has to be. I also warn my students never to try to fool their bosses, never try to slip something by. Always be on your employer’s side; never act on another agenda.

Making sure that goals are identical is one reason for doing as much research as possible “in house.” Another is that your own experts will always know more about your situation than will outsiders. Yes, there are times when bringing in someone makes sense—but generally just do so to evaluate what has been done locally, and only if the outsider has been carefully vetted.

There are other reasons for doing one’s own research or, at least, keeping it local and open. As anybody who has attempted real research knows, the value of conducting research isn’t only in finding what you are looking for, but in the other things you discover along the way. Narrowly directed research, without the possibility of accidental findings, generally does little but confirm what we already “know.” This, of course, is why database searches can only be a small part of any research project, a ‘review of the literature’ at most.

Even then, the search needs to be broad, covering as much ground as possible. Just finding an article or two is never enough—nor is using a single database: there is no ‘one stop shopping’ in library research.

I saw a website today, for something called PolicyDirect, saying it is “connecting postsecondary education research with decision makers.” At the bottom of the page is a paragraph starting with this: “PolicyDirect serves as a one-stop, easy-to-use resource for quality research that illuminates critical findings and further challenges around important student outcomes.”

Aside from language that seems more like a smokescreen than illumination, what bothers me is that there probably are higher-education “decision makers” out there who would be grateful, without questioning it, for such a service. What bothers me even more is that this completely bypasses the century-old concept of shared university governance. That is, it should be assumed that the “decision makers” include the faculty—who just happen to also be the primary movers of postsecondary research. The research and the ability to connect to it is already there. Why establish another pathway? But that’s a topic far larger than I can tackle in a blog post, so I will keep to something simpler.

What bothers me, too, is that PolicyDirect does not even represent best practice in business. It asks the “decision makers” to rely on the judgment of outsiders whose purposes may be far from those of the very “decision makers.” On its “About Our Reviewers” page, PolicyDirect says:

Selection of the Academic Fellows were based on the fellows. [sic] promising body of research and interests, recommendations from senior scholars in the fields of higher education and public policy, as well as input from national philanthropic leaders. This prestigious selection allows a unique opportunity for the fellows to influence the national postsecondary education agenda by evaluating critical research to elevate the current policy discourse. New Fellows are selected each year, in partnership with former Fellows in order to preserve consistency.

Nowhere is any indication given about who these “fellows” actually are. Maybe they are listed on the pages of the Institute of Higher Education Policy (IHEP) or Lumina Foundation pages (for these are the sponsors of the site), I don’t know, but nothing at PolicyDirect tells anything about them. “Decision makers,” it seems, are supposed to simply trust that these “fellows” are impartial, that they aren’t steering people toward specific articles or types. My students, who now know something about the possible follies of research, would be appalled.

In addition, the articles are presented through “excerpts,” not abstracts or summaries of some other type. Strangely enough, the site even mentions “full excerpts,” an odd phrase….

Just to see what PolicyDirect is doing, I tried a number of searches. One thing I noticed immediately was that a single article kept coming up at the top, a piece by Clifford Adelman who, it turns out, is a “Senior Associate” for IHEP.

At some point, I decided to compare PolicyDirect results with Google. Wanting  to use something neither too common nor too obscure, I searched on “Fred Keller Personalized System of Instruction.” PSI is something I know about but that has not been part of education discussions for quite some time. From Google, I came up with 180,000 hits; from PolicyDirect, 100. The first one from Google links to an .edu site with a .pdf specifically on PSI. All of the following three or four pages worth dealt directly with PSI.

On PolicyDirect, after that same Adelman article, which doesn’t mention Keller at all, came 99 hits. None of them seem to have anything to do with Keller or with PSI.

Thinking I had perhaps been too esoteric, I tried another search, on “early college high school,” a topic that should be of interest to PolicyDirect, given its connections to IHEP and Ilumina. Google gave over 400 million hits, each on the first few pages directly relevant. PolicyDirect?  99, if you count that ubiquitous Adelman article. Here, at least some of the articles did pertain to the topic, though many seemed a little far removed. Little of it seemed like it would be helpful in developing an understanding of the ‘early college high school’ movement.

My question, through all of this, is what’s the point? I quickly established that I can get more pertinent results through Google than through PolicyDirect, so why would I want to use PolicyDirect? Especially since I have no idea what the databases are that PolicyDirect has searched in order to prepare its own “excerpts,” and to what ends, I really cannot trust what I am finding there. I mean, Mr.PolicyDirect, just why are you trying to help me? I can carry my own bags—and if something is missing, I have only me to blame. Why should I trust you, a complete stranger, to do my work for me?

I hope our “decision makers” are asking just this as they look at the PolicyDirect site. My students, who are right now working on guides to effective web research, certainly would be.