Open innovation has been around for over a decade now, and it’s easy to fall into the trap of believing that the metrics involved in it are rather straight forward. I mean there is often a very clear output that comes from the process, so you can easily subtract the costs from the benefits to get an ROI, right?
Yes, and no, and there is a growing amount of literature tackling the issue of measurement and metrics. A recent paper published by Finnish researchers explored the literature on the topic. The paper complains about the lack of clear measurement approaches within open innovation, and attempts to provide a unification of metrics, in particular of a quantitative kind.
Their analysis suggests metrics such as the number of ideas, number of participants and technology acquisition are viable metrics to understand the success of open innovation. The paper is let down however by a pure focus on outputs.
An alternative approach has been provided by a second paper emerging from the University of Aachen, in Germany. It suggests that three principles should underpin any measurement process:
- Use a separate metric for each approach to open innovation
- Define metrics for inputs, processes, outputs and outcomes
- Use your metrics in a way that is instrumental (ie used in decision making), conceptual (ie it provides general enlightenment) and symbolic (ie when used to justify existing decisions)
They arrived at the principles after surveying over 100 large European companies across multiple industries. Their final measurement scorecards are split across the various types of open innovation approaches, with up to 14 metrics supplied with each. Each scorecard has various common elements however, including sections for initiation, implementation and metrics for inputs, processes, outputs and outcomes (as highlighted above).
Interestingly, amongst all survey respondents, the top metric for their own open innovation projects was around the commitment to the project by management. It underlines how fragile not only innovation is in general, but specifically how uncertain things remain for open innovation as a concept.
The report is interesting in that it provides some useful metrics for every stage of the open innovation process rather than simply focusing on the outputs. With innovation often struggling to overcome political resistance internally, this is a valuable process to undergo.
Of course, another aspect of open innovation that is not mentioned in the paper concerns ones intellectual property, and how open the organization is with that. A Dutch study explored this very topic, revealing that companies that tap into external sources for research and development perform significantly better on innovation than those that keep everything in house.
Researcher Luca Berchicci from RSM Erasmus University found that there is an optimum point, beyond which there is no further gain to be had from opening up to the crowd. He believed that sweet spot to be around 34% of investment. Whilst it would be foolish to assume that R&D budgets should be radically re-drawn, it does show that many organizations are continuing to under-utilise the crowd in their innovation efforts.
Whilst ideation has become a popular use of open innovation, there is as yet little evidence that organizations are using crowd based insights further along than that, for instance when the idea is in the development stage.
It can appear tricky to measure innovation, and indeed a previous study from the Boston Consulting Group found that the overwhelming majority of companies value their efforts by how much they spend rather than the output they obtain. These papers should begin to give you a better idea of the kind of things you can measure to get a better gauge of how well you’re doing.
