When it comes to gauging public opinion, policymakers have increasingly been relying on social media to get real-time feedback on their work. While this has undoubted temptations, research from Dartmouth suggests that social media may not provide a representative sample.
Unlike surveys, which aim to gather opinions from different kinds of people that are similar to the overall population, it’s important to know that social media doesn’t accurately represent everyone.
Not representative
For example, more young folks use social media than older folks. Less than half of people aged 65 and older use social media, but more than 80% of those under 50 use it, according to data from 2021.
And, different social media platforms have different types of users. Younger people tend to use Snapchat and Instagram, while older people use Facebook more. These differences in who uses the platforms can make the information we get from them a bit biased.
There’s also something called “participation bias,” which means that not everyone who’s on a social media platform actively joins in conversations. It depends on what’s being talked about.
“Even if you have everyone on Twitter, they may only participate in certain topics—ones that they find interesting or maybe feel comfortable talking about in public,” the researchers explain.
A vocal minority
The researchers explain that when a small group of people talks a lot about a specific topic on social media, their opinions can dominate the conversation.
While researchers have looked at participation bias in traditional surveys, they haven’t studied it in the digital world much. To figure out how much participation bias there is, the researchers made a computer program.
This program looks at social media data and, using information from surveys that represent the general population, guesses what kinds of people are taking part in online discussions. The difference between what the computer program predicts and the actual social media user demographics tells us how much bias there is for that topic.
In their research paper, they looked at the topic of gun control in the United States. They compared data from a social media platform, which they called “X” (it was Twitter), with survey data from places like NPR, Newshour, and Marist.
Uneven spread
According to data from Pew (a research organization), Twitter has about the same number of men and women users, and most of its users support the Democratic party. However, when it comes to talking about gun control, the computer program suggests that more Republicans and men are taking part in these conversations on Twitter.
“We’re hoping that this kind of research can help put what we see on social media into context, and also to make it easier to track changes in public opinion without the need to run repeated, expensive surveys,” the researchers explain.
The computer model is made to handle the messy data on social media, like posts from automated bots. But it can only work well when there’s survey data to compare it to.
Surveys take a lot of time and money to do, and lately, fewer people want to take part in them. Digital data can help policymakers if they can figure out how to deal with the problems and biases in that data.
