Despite most innovation these days recombining old ideas in new ways, there remains the perception that innovation is a perpetual hunt for new and whizzy ideas.

A new study from the Stanford Institute for Economic Policy Research suggests that truly novel ideas are not only harder and harder to come by, but they tend to be ever more expensive to explore.

“The thought now of somebody inventing something as revolutionary as the locomotive on their own is inconceivable,” the authors say.

Research productivity

The authors strove to examine research productivity on an aggregate, national level, as well as within the technology, medical research and agriculture industries specifically.

The analysis finds that whilst research spending is going up considerably, the ideas output by each researcher is going down considerably.  They suggest that this huge increase in research inputs has helped the American economy to maintain growth as this increase has offset the decline in productivity.  This is reflected in the number of people engaged in R&D, which has mushroomed twentyfold since 1930.

“It’s getting harder and harder to make new ideas, and the economy is more or less compensating for that,” they say. “The only way we’ve been able to roughly maintain growth is to throw more and more scientists at it.”

With R&D such a fundamental driver of economic growth, the findings are a worry, as it means the economy has to double its spending on R&D every 13 years just to maintain the same rate of growth.

Exponential growth

We’re living in supposedly exponential times, with technological progress growing rapidly.  The paper suggests that the main exponential growth trend is in research investment though rather than research output.

Indeed, Moore’s Law itself is a fine example, as whilst processing power has doubled roughly every two years since Moore proposed it in 1965, the authors report that the investment required to maintain this growth has risen by a factor of 78 since 1971.  Or if you want to think of it another way, it takes 75 times as many researchers to continue with Moore’s Law today as it did in the early 70s.

“The constant exponential growth implied by Moore’s Law has been achieved only by a staggering increase in the amount of resources devoted to pushing the frontier forward,” the authors say.

Such falls in research productivity are not confined to technology, with industries such as agriculture also experiencing a decline.  Average yields across the four main crop types doubled between 1960 and 2015, which is great, but the research required to achieve that growth increased considerably in that time, with some crops requiring 25 times the investment to maintain yield growth.  Across the board, the paper says that research productivity in agriculture fell by up to 6% per year.

Similar patterns emerged in the pharma industry.  The researchers compared the expenditure on each new drug with the increase in life expectancy to determine the productivity of medical research.

The empirical findings on breast and heart cancer suggest that at least in some areas, “it may get easier to find new ideas at first before getting harder,” the paper stated.

This trend is also evident across the stock market, with some 85% of firms showing a clear decline in research productivity, as measured by growth in sales, market capitalization, employment and revenue-per-worker productivity, even whilst spending on R&D rose considerably.

This decline was, on average, 10% a year, thus taking 15 times the expenditure to achieve today what would have been possible 30 years ago.