Whilst it’s tempting to think that scaling ideas and innovations is an exact science, the reality is that it’s an inherently uncertain endeavor. That doesn’t prevent us from trying to understand if there are common factors that underpin those ideas that thrive however, and that was the aim of a recent study that explored over 5.3 million life science research papers published between 1970 and 1999.
They identified the areas covered in each of the papers, together with any that covered a combination of areas, with the intention of exploring how these ideas compete for the attention of the academic community.
They hypothesized that if an idea emerges in an already congested field, it can easily get lost in the mass of other research being published. The key therefore was to publish in a relatively uncrowded space.
Key to success
The data revealed two key aspects of success for any idea. The first was that novelty was not necessarily helpful. Indeed, the more novel and unusual an idea was, the less likely it was that it would be picked up by someone else and worked with. People need to have some level of familiarity or conformity to what has gone before to work from with any new idea.
The second key finding revolved around the fight for attention mentioned above. If an idea space is heavily contested then it can be difficult to make your own work stand out. People searching for work in a particular domain have a finite amount of attention, so spreading that attention very widely can render if much less likely that it will fall upon your specific work.
The findings are interesting because we often assume that the best innovations have to be the most novel ideas, and whilst that may be true in terms of the breakthrough itself, if an idea is ‘too’ novel, then it harms the spread of it.
Attention bias
That is reasonably well known from previous research, but what is especially interesting from this paper is the focus on attention. The study obviously examined the scientific landscape, but the message is relevant across disciplines.
It’s well known that most innovations today are what’s known as recombinative. In other words, they take concepts that already exist and tweak them or apply them in different ways. For this recombination to happen, initial innovations need to be findable, so operating in a crowded space can make this harder than it might be in a less crowded environment.
The researchers believe that an understanding of these two biases can help researchers and innovation managers better understand the landscape and adapt their behaviors accordingly. They also highlight the possibility for technologies such as artificial intelligence to lend a hand.
For instance, a recent paper from researchers at Carnegie Mellon University and the Hebrew University of Jerusalem highlights an AI driven approach to mine databases of patents and research papers for ideas that can be recombined into solutions for new problems.
The research used analogies to help train an algorithm to identify seemingly disparate methods and problems across the intellectual databases to hunt for potential innovations.
“After decades of attempts, this is the first time that anyone has gained traction computationally on the analogy problem at scale,” the authors say. “Once you can search for analogies, you can really crank up the speed of innovation, If you can accelerate the rate of innovation, that solves a lot of other problems downstream.”
The team believe that the approach could easily be scaled up to be used as part of the innovation process by allowing organizations to find previously hidden connections between patents or research papers. With both datasets enormous and growing at a tremendous rate, it’s a problem that well suits autonomous methods, so it will be interesting to see what comes next for the team and their system.
