How to avoid the contributor bias in crowdsourcing

connected-communityAs anyone that’s ever managed an online community will tell you, there is nearly always a small core of users who provide the bulk of your input.

Whether it’s Wikipedia or citizen science, crowdfunding or discussion forums, you will find that a small section of your userbase are doing most of the work.

It has meant that the trusty 90:9:1 heuristic has remained remarkably stable as communities have evolved online in recent years.

Whilst there is a degree of inevitability to this, a recent paper explored whether it was healthy for the community.

Participation bias

The study looked at the Open Street Map, which has 1.2 million users, but just 15% of those produce the vast majority of the information available via the platform.

The analysis showed that the kind of content itself was not hugely different between the power users and the more fleeting participants.  When they analyzed who those users were however, very clear divides emerged, with affluent, urban areas gaining significantly more coverage than less affluent parts of town.

Such biases are particularly strong on projects that require sustained input from users.  Humanitarian efforts, such as those studied on Open Street Map often attract intense bursts of activity to begin with, before dropping off considerably, thus leaving a small group keeping things going.

The importance of transparency

It’s worth noting that there is no suggestion of deliberate bias on behalf of the users, so the authors suggest a good first step is highlighting some of the areas not being served by the current userbase.

Exposing this apparent weakness did seem to encourage users to step in and fill the gap.  Processes can also be designed to reduce some of the barriers to participation in certain categories.

For instance, contributions could be shrunk into smaller chunks to allow more people to participate.

The issue of securing a wide and varied community of participants is something that is likely to prove an ongoing challenge.  You can listen to Professor Licia Capra from UCL talking about the research undertaken at Open Street Map in the video below.

Let me know your thoughts and whether you have any good tips for keeping users engaged in your project.



Leave a Reply

Your email address will not be published. Required fields are marked *