The Misinformation of Crowds

2008/10/17 · by · in Idiocracy, Punditry, Tech*

I’ve seen much recently piggy-backing on the Wisdom of Crowds. Much of the Web 2.0 seems highly focused on exploiting this phenomenon:

  • Datamining the crowd: Yahoo and Google both have an Answers service that purports to harness the wisdom explicitly.
  • Ranking results: Digg, Newsvine, Technorati all use forms of distributed social voting to determine quality.
  • Finding the Diamond: StackOverflow uses ranking to filter the ‘most useful’ answers to the top.
  • CrowdSorcing (use the crowd to do your work): Image Search has been turned into a labeling game
  • Web Search is dominated by Sergey and Brin’s PageRank algorithm which uses hypertext links as the voting system.
  • Google’s new Book Search (quotation finder), which is geared to work across languages, correlates text samples across different media, counting the number of occurrences of a phrase.
  • Prediction Markets, fundamental Democracy, etc…

I’m sure that there are many more examples that I’ve neglected to mention.

Unfortunately, the effectiveness of Crowd Wisdom relies heavily on some assumptions which may prove shaky in the real world:

  • Independence of participants. The crowd is wisest when it’s composed of independently thinking individuals. Lemmings and mobs make very foolish crowds, aka herds. If each of the participants uses the popular reasoning “what’s good enough for everyone else is good enough for me,” we have a crowd with very many inter-dependent voices.
  • Diversity of Opinion. The crowd is wisest when it can draw upon widely varied knowledge and experience.

Wikipedia readily identifies what I see as the fatal flaw:

Surowiecki studies situations (such as rational bubbles) in which the crowd produces very bad judgment, and argues that in these types of situations their cognition or cooperation failed because (in one way or another) the members of the crowd were too conscious of the opinions of others and began to emulate each other and conform rather than think differently.

So, what happens then when the world’s knowledge is aggregated by these algorithms? Our collective knowledge becomes stricken by a dangerous positive feedback loop. Due to time constraints we prefer superficial syncophantic opinions to in-depth analysis. Everyone will use the results of these tools to form their own opinions, which undermines the assumptions the tools are built upon. Because of this self-inconsistency the Wisdom of Crowds algorithms should be looked upon with reservation and skepticism.

The Wisdom of Crowds is too unstable and should find itself becoming the tool for the Misinformation of Crowds.

Aside: Already we have evidence of real-world damage caused by such algorithms and social psychology. United Airlines stock crashes because Google News posted a six-year old story about bankruptcy as if it were current.