Using big data for ideation

Last summer, a Japanese company made the news after it announced that an AI would be joining its board.  The AI was deployed to mine a large database of previous creative projects to suggest possibly new creative directions for an advert.

It was a story I was reminded of when researching Proto Ideation, a new platform developed by researchers at Columbia Business School.  The researchers behind the platform conducted an interesting study into the ideation process, and in particular how we judge the creativity of an idea.

The hypothesis they tested was that we gauge ideas as being more creative based upon the balance between their novelty and familiarity.  They mined thousands of ideas generated via the Proto Ideation platform to try and discern any patterns.

“You can think about idea generation as creating a recipe where you to choose the appropriate ingredients and mix them together” the researchers say. “By analyzing the text in a large number of ideas across different domains, we were able to link an idea’s judged creativity to its set of ‘ingredients.’ We found that what makes an idea creative as judged by both consumers and firms’ executives is a mix of ingredients (words) that includes a balance between words that commonly appear together (familiar combinations) and words that do not (novel combinations).”

Creative ingredients

So, creativity is not purely a result of the novelty of an idea, but rather an ideal combination of novelty and familiarity.  The results from the experiment helped them in the development of their ideation platform that is designed to help people come up with better ideas.

“The tool analyzes in real time word combinations included in the idea and recommends words that help the innovator improve her idea” they say. “If the idea is too familiar the algorithm will offer words that would make it more novel. On the other hand, if the idea is too novel, it will offer words that will make the idea more familiar.”

The tool was found to help people with their ideation, especially in helping improve the novelty of ideas that had slipped into familiarity.  The results suggest that big data can play a role in the creative process, whether it’s in fully automating things as in the Japanese example I started the post with, or in providing a level of support such as in this latest example.

“By leveraging tools from the world of Big Data such as text mining and semantic network analysis we were able, for the first time, to scientifically analyze a large set of ideas and identify successful patterns that can help people improve their ideas. In a way, our tool turns the computer into a “sous chef” that helps the user assemble the right ingredients to combine in order to form creative ideas,” the researchers conclude.

It will be interesting to see just where big data and AI go next.  If you’d like to play around with the tool, it’s open to the public, and can be accessed via the link at the start of the post.



Leave a Reply

Your email address will not be published. Required fields are marked *