Social media has often been regarded as a wholly imperfect means of understanding any event or community, if for no other reason than a relatively small proportion of any community will be using the platform.  Nonetheless, new research from Stanford University advocates using Twitter and AI to reveal the psychological health of a community.

The researchers acknowledge that Twitter doesn’t provide a representative sample of the population, but nonetheless believe it’s useful.  The researchers analyzed around a billion geo-tagged tweets sent between 2009 and 2015.  The tweets were compared to 1.7 million responses sent to the Gallup-Sharecare Wellbeing Index.

Surveys such as that done by Gallup have historically been a central part of attempts to understand a population’s wellbeing.  They are accurate, but require a lot of time and money to undertake, with some surveys taking years to garner a robust response.  The hope is that data from social media can help augment this data and alleviate some of the burden involved in collecting survey data.

Reliable data

The researchers trained an algorithm to assess both the responses to the survey and posts made on social media from the same people to try and understand key similarities in style and content.  This has traditionally been difficult, as online slang, such as LOL, might ordinarily be seen as a positive expression, yet is also associated with lower income areas.

Similarly, words such as ‘homework’ and ‘taxes’ can appear negative, yet are also commonly associated with higher income areas.  As such, it’s important when using language to measure wellbeing that these cultural differences are understood.

The machine learning algorithm helps to do that, and the AI found that phrases such as LOL were not a good indicator of our wellbeing, and instead proposed words such as ‘fun’ and ‘excited’.

“Having the computer learn the words may be the best way to find words that measure well-being,” the researchers say. “Differences in language use can be quite complex.”

This is important, as wellbeing is a complex thing, and can be associated with a wide range of other factors, including our overall health.  For instance, the researchers note that stress or depression are strongly linked to excessive drinking or smoking, which have clear implications for overall health.

They also believe that the approach they’re taking could be useful in such fast moving crises as the current COVID-19 pandemic, and help provide researchers with real-time insights into the health of a community.

“COVID-19 is a natural disaster that interrupts our social norms and routines at an unprecedented scale,” they conclude. “With this real-time Twitter-based technology, psychologists can monitor if loneliness and anxiety are taking hold in communities, and how our well-being is impacted by social distancing.”