Britannica upset with Nature

A while back, Nature posted a comparison of the free encyclopaedia wikipedia with that of the formal published Britannica. It appears that the general conclusion Nature reached, which was that wikipedia was sometimes as accurate or even better than Britannica on scientific articles has struck a nerve. Nature begin their response pointing out what they were accused of:

In our issue of 15 December 2005 we published a news article that compared the Internet offerings of Encyclopaedia Britannica and Wikipedia on scientific topics (“Internet encyclopaedias go head to head”, Nature 438 (7070) p900-901; http://dx.doi.org/10.1038/438900a). Encyclopaedia Britannica has now posted a lengthy response to this article on its website, accusing Nature of misrepresentation, sloppiness and indifference to scholarly standards, and calling on us to retract our article. We reject those accusations, and are confident our comparison was fair.

The original analysis didn't seem to have been flawed in any particular manner and seemed to have been perfectly fair, though I feel Britannica does have a few legitimate complaints in places. Firstly, Nature did only take smaller segments from articles to show their reviewers and often some of the complaints were addressed in the larger body of text. Additionally, Nature seems to have taken material that is from other editions of the encyclopaedia, which are intended for younger audiences or not included in the main edition of Britannica. I do see that as a valid complaint because that would skew the analysis. After-all Nature are implying the wiki gets as accurate as the encyclopaedia Britannica, which is a somewhat different thing than their other publications.

Otherwise, through reading both the responses from Nature and Britannica, it does seem that Britannica were trying to save face and their reputation primarily as many of their complaints, such as shortening articles, were equally applied to wiki articles by Nature. Of course this defensive response from Britannica does make sense, who would want to be beaten out by a free online encyclopaedia that's not even written by experts? This point in particular seems to have got Britannicas goat, as you can see here from the opening of Britannicas response:

In its December 15, 2005, issue, the science journal Nature published an article that claimed to compare the accuracy of the online Encyclopædia Britannica with Wikipedia, the Internet database that allows anyone, regardless of knowledge or qualifications, to write and edit articles on any subject.

Which then moves onto a typical attack that the wiki often hears from its detractors:

Wikipedia had recently received attention for its alleged inaccuracies, but Nature’s article claimed to have found that “such high-profile examples [of major errors in Wikipedia] are the exception rather than the rule” and that “the difference in accuracy [between Britannica and Wikipedia] was not particularly great.”

Arriving amid the revelations of vandalism and errors in Wikipedia, such a finding was, not surprisingly, big news. Within hours of the article’s appearance on Nature’s Web site, media organizations worldwide proclaimed that Wikipedia was almost as accurate as the oldest continuously published reference work in the English language.

This is often a problem with the wiki, particularly on controversial issues like evolution, the holocaust and other contentious articles that are prone to vandalism. Many of the articles on wikipedia are of a very high quality however and are well maintained by individuals with expertise in their fields. Unfortunately, the wiki seems better for "fan cruft" and other similar information like details about popular computer games or anime series than as a scholarly resource at the moment.

In any particular event, I think that Nature probably weren't as careful with their analysis as they could have been originally. Some of the mistakes that Nature claim to have identify in the Britannica articles do not appear to be overly relevant at times (see Britannicas response for some details). Without seeing what the same reviewers thought about the relevant wikipedia articles, it would be impossible to determine if there was any real reviewer bias either way. Hopefully Nature will produce a more detailed response that clarifies how their reviewers went about determining errors and the like. Importantly, I would like to know how Nature decided what parts of the articles to take as that could heavily impact on the end result.