There has been an unquestionable rise in appreciation for collective intelligence in recent years, as organizations of all types have sought to tap into the wisdom of crowds.
When James Surowiecki first coined the term, he identified four criteria by which crowds became wise, with most of the conditions battling the emergence of groupthink.
World Without Oil
A recent paper has explored this issue via the construct of a game designed to prompt players to think about a world without oil.
The game, called World Without Oil (WWO), was played both online and offline during 2007 and was intended to test out various peak oil style simulations.
Analysis of participants found that participation levels were very similar to other crowdsourced projects in terms of contribution levels. The vast majority made but a handful of contributions, with the bulk of input derived from just over 10 percent of the player pool.
Therefore, the content was dominated by a small core of regular players that goes against the notion that opening things up to the crowd guarantees thought diversity.
This was reflected in concerns raised about the credibility and breadth of inputs into the game. The majority of solutions proffered were individually orientated rather than anything systemic in nature, and there was a lack of credible authority to validate many of the suggestions, with doubt raised about how effective the peer review system used in the game really was.
This inevitably led to a shallower pool of ideas from which to choose from, and a lack of anything truly innovative emerging from the process.
The authors suggest that whilst the game performed reasonably well in getting participants to engage with the challenge, the range of content produced was all rather narrow and reflected a sense of groupthink amongst the participants.
As a result, the authors believe that the game largely failed in its mission to create a discourse around sustainability that covered the full gambit of issues on the topic, such as social justice, health and wellness, and security and governance, all of which were largely absent from the discussions.
Instead, there was a sense that contributors were judged on the weight of their contributions rather than on the value of them. This, coupled with the challenges in bringing factual data into the game, led many players to internalize perspectives that were false and misleading.
It provides a telling reminder, should one be needed, that crowdsourcing should not be regarded as an easy to implement panacea to the challenges we face. If it is to deliver the kind of insights that we hope for, then it needs to be constructed in the right way to ensure that the thought diversity uncovers innovative insights.
As Surowiecki says, such efforts need to have:
1. Diversity, and not just in terms of the knowledge people come to the party with but the sources they get subsequent information from.
2. Independence, so that the opinions of each individual contributor are not unduly influenced by their fellow participants.
3. Decentralization, so that decisions are made locally based upon local knowledge and insights.
4. Aggregation, so that the wisdom of individuals is collected and pooled together into something valuable. This is often where things fall down as you get ‘design by committee’. I’ve written before about the value of having someone with exceptional organizational knowledge who can act as the aggregator pooling together the thoughts and insights from the diverse range of sources. Aggregation seems to be the most tricky of the four conditions to satisfy because there are so many different ways to aggregate opinion, not all of which are right for a given situation.
It’s clear that the World Without Oil game failed on many of these conditions, and therefore failed to materialize the kind of insights the makers no doubt hoped for.
Hopefully your own crowdsourcing efforts won’t fall into similar difficulties.
