When groups are smarter than the experts in them

At a county fair in 1906, a statistician named Francis Galton asked the crowd to guess the weight of an ox. He expected to demonstrate the superiority of expert judgement. The crowd's average guess turned out to be closer to the true weight than any individual estimate, including those of livestock specialists. James Surowiecki uses this story to open a sustained argument: that groups, under the right conditions, are consistently smarter than the smartest people within them.

Those conditions matter enormously, and Surowiecki is admirably clear about what they require. For a crowd's collective judgement to be reliable, he identifies four necessary conditions: diversity of opinion among participants; independence so that individuals are not simply copying each other; decentralisation so that local knowledge can feed into the aggregate; and a mechanism for combining individual estimates into a single collective answer. Remove any one condition and the crowd can become spectacularly and quickly wrong.

The failures are as instructive as the successes. Herding — the tendency to follow what others appear to be doing rather than using your own information — destroys diversity and independence simultaneously. Stock market bubbles, corporate groupthink, and cascading bank runs all share this structure: once enough participants are watching each other rather than the underlying reality, the crowd amplifies error rather than correcting it. The same mechanism that produces wisdom can produce catastrophic collective delusion.

Surowiecki's examples range from prediction markets to the military search for a lost submarine to the consistent accuracy of betting odds relative to the assessments of individual expert pundits. He writes as a journalist who has spent considerable time observing financial markets and he is genuinely interested in the practical question: when should you trust markets, expert committees, or democratic votes, and when should you be sceptical of all three?

The book is ultimately about information and how it moves through groups. When the conditions for collective intelligence are properly met, aggregation is a genuinely powerful tool — more powerful, in certain domains, than any individual expertise. When those conditions are violated — when conformity replaces diversity, or authority displaces independence — the mechanism that produces wisdom produces groupthink instead, often with considerably more confidence.

The Wisdom of Crowds was published in 2004 and anticipated many of the debates about prediction markets, online aggregation, and collective intelligence that followed over the next two decades. Some of its enthusiasms about financial markets have aged less well than the underlying theoretical framework, but the core argument about the conditions under which collective judgment reliably outperforms individual expertise has been repeatedly supported by subsequent experimental work. The book is the most accessible account of that argument and, read carefully, it is also a lucid warning about exactly when and why the same mechanism goes catastrophically wrong. The book is the most accessible account of that argument and, read carefully, a lucid warning about exactly when and why the same mechanism that produces wisdom produces catastrophic collective delusion instead. The book is the most accessible account of collective intelligence available and, read carefully, a warning about when it fails.