Whilst Steve Jobs famously decried the value of focus groups and other mechanisms for gauging the feelings of consumers, they remain a popular tool for many companies. They remain something of a blunt tool however, and new research suggests that AI could do a better job, both in terms of the efficiency but also the quality of consumer insights delivered.
The research, which was conducted by a team from MIT, utilized machine learning to make sense of user-generated content, such as reviews, social media and blogs to provide an insight into the needs, preferences and attitudes of consumers.
This has been an option for many years, but concerns have been raised about the quality and reliability of social media data. The volume of data also renders it almost impossible for humans to make sense of.
Artificial assistance
“As more and more people turn to the digital marketplace to research products, share their opinions, and exchange product experiences, large amounts of UGC data is available quickly and at a low incremental cost to companies,” the authors explain. “In many brand categories, UGC is extensive.”
On Amazon, for instance, there are around 300,000 reviews in the health and personal care category alone. To test the ability of AI to make sense of such a large and constantly changing dataset, the researchers constructed a custom dataset for oral-care products based upon Amazon reviews.
Their system was able to accurately identify customer needs purely from the reviews left on Amazon. The system firstly identified the relevant content and stripped out redundant data, with the processed data then analyzed by humans to formulate the needs of customers from the smaller sample.
“In the end, we found that UGC does at least as well as traditional methods based on a representative set of customers,” the researchers explain. “We were able to process large amounts of data and narrow it to manageable samples for manual review. The manual review remains an important final part of the process, since professional analysts are best able to judge the context-dependent nature of customer needs.”
