More Wikipedia bashing

If it wasn't so frustrating, it would be amusing to see Stephen J. Dubner of Freakonomics and his frequent efforts to bag Wikipedia.

He's taken up prefacing every attack on Wikipedia with a explanation of just how much he likes it, no really, he likes it just fine. This time, he gloats that the US Patent Office has removed Wikipedia from their list of approved sources of information when they have to check a patent application. Apparently, the US Patent Office has, up until now, been using Wikipedia as an official source "for years", according to critic Greg Aharonian.

I smell a rat. Wikipedia wasn't even close to ready for prime time as recently as two years ago; it's arguable whether or not it is ready now (although I would argue that, used with care, it is). It certainly wasn't heavily in the public eye two years ago. I find it difficult to credit that an organisation as conservative as the US Patent Office would have been using Wikipedia in 2004.

But what the hey, maybe it was an official source. Stranger things have happened.

Dubner writes:

The argument in defense of Wikipedia that I find most troubling is that it is self-correcting and self-policing, which is to say that, Hey, in the end all the mistakes and vendettas get fixed by caring and level-headed people.

If being self-correcting and self-policing (rather like the free markets Dubner champions as an economist) is a bad thing, what's the alternative? An official government fact-checking agency? We could call it The Department of Homeland Truth, and make sure that only things that get approved by the bureaucrats in the government are printed.

Perhaps not.

Maybe Dubner thinks that the self-correcting model is a bad one, and that the correct model is something like that of the Encyclopaedia Britannica, which is ... self-correcting and self-policing. Hmmm.

So what is his point? As an economist, he surely knows that all of science (including the Dismal Science, economics) is self-correcting and self-policing, and operates by concensus rather than fiat.

Science and knowledge advances according to what is said, not by who says it. Britanica has got a reputation to uphold, and it may be that we can trust that they value their reputation enough that we should treat the facts they publish as trustworthy; but ultimately we have to take their word for it. In other words, we believe Britannica because of who they are. Wikipedia instead exposes the whole messy business of deciding what's true and what isn't, and that is far more valuable, even more valuable than the facts themselves. If Wikipedia says something is a fact, we can (if we take the time and effort) see exactly why we should believe it.

Dubner has made an empirical claim here. He stated:

The problem, of course, is that if someone happens to read or cite a Wikipedia entry at a moment when all those things haven’t been fixed, which is obviously a vast, vast, vast majority of the time, then the mistakes get promulgated as fact.

Dubner claims that, on average, supposed "facts" in Wikipedia are wrong not just the majority of the time, not just the vast majority of the time, but the vast, vast, vast majority of the time.

I'm not sure what the difference between "vast majority" and "vast, vast, vast majority" would be. 51% is clearly "the majority", and it stretches the common meaning of the words "vast majority" to apply it to anything less than a four-fifths majority. Three vasts surely means something like 999,999 in a million, but let's give Dubner the benefit of the doubt and make it a nice low 90%. So Dubner believes that, overall, any supposed fact on Wikipedia will be wrong at least 90% of the time.

That's an empirical claim that can be tested. Choose (say) one thousand random Wikipedia entries, and count the number of supposed facts in those entries. If more than one in ten are correct, then Dubner is full of sh.. mistaken.

Of course, not all Wikipedia entries are of equal quality. I dare say that there are obscure "facts" on pages that have never been checked since they were created that are wrong; no doubt there are entries with all the trustworthiness of gossip magazines. So what? Lazy and incompetent researchers will treat bad sources as if they are good sources, Wikipedia doesn't change anything there. Ten years ago, people used The Little Bumper Book Of Fun Facts For Tiny Tots as their major reference, and now they use the Internet the same way. Bad reseachers will treat anything they read as infallible, not just Wikipedia. If you are doing "real" (i.e. important) research, you should not be relying on any single source of information, whether it is Wikipedia or Britannica, because no single source is error-free. Good researchers will treat Wikipedia as it is meant to be used: look for citations, and follow the link to verify the facts. And unlike most other secondary sources, Wikipedia warns you when caution is needed.