Research reveals that managers suck at judging new ideas

rating-ideasI wrote recently about the value in having leaders who are confident in their creativity, after research found that such confidence trickles down to their team, who have higher creative output when working under such a leader.

Alas, that still leaves a question mark over whether those leaders actually know a good idea from a bad one.  History is littered with examples of fantastically successful innovations that were rejected at the first time of asking.

All of which begs the question, who exactly is best placed to judge the potential of an idea?  Traditionally we would take a polemic view of the matter and think in terms of either the inventor themselves, or the manager charged with backing that project.

A recent study from Stanford researchers suggests that we need to reassess that thinking.

Creative forecasting

The paper suggests that in actual fact, neither the inventor or the manager are best placed to assess the potential fortunes of a new idea.  Instead, the best judges appear to be peers of the inventor.

They’re a good judge because they’ve often spent time exploring and generating their own ideas, even if not the specific idea in question.  That makes them well placed to appreciate the process involved.

The author gleaned his inspiration from the circus industry, where performers would regularly come up with new and innovative acts.  The industry typically differentiates between those who come up with the ideas, and those who assess the ideas for inclusion in future shows.

In other words, the managers act as a gatekeeper between the performers and the audience.  It’s a similar setup to how innovation works in many organizations.

Inside innovation

The authors collated videos of some 150 circus acts from around the world.  These were analyzed by a cohort of nearly 340 industry professionals, consisting of both managers and performers.  They were tasked with predicting how successful each performance would be.

These predictions were then tested against reviews of each performance by a pool of 13,000 audience members.

Perhaps not surprisingly, creators themselves were generally bad judges of the potential of their idea – they were simply too close to it to be objective and tended to overestimate its chances of success.

When those same people were asked to evaluate the ideas of other people though, their success rate went up considerably.

The managers were usually rather hopeless evaluators, with a tendency to favor the conventional over the novel, which would have caused them to miss out on some of the most successful acts.

Past success is not indicative

This was fascinating because most of the managers were performers themselves in a previous life.

“One might think that if you’re a successful creator, you have good taste, and when you get promoted to a manager role, it stays with you,” the authors say. “But this research suggests that the creator role may promote good taste, while the manager role may undermine it.”

So why was this?  The authors suggest that it’s because creators focus their attention on new ideas, whereas managers are focused more on evaluation.  So one is engaged in divergent thinking, whilst the other is engaged in convergent thinking.

To successfully evaluate ideas you need both modes of thought, but managers typically only utilize convergent thinking, which focuses attention on the past rather than the future.

Dual thinking

This hypothesis was then tested in a lab environment, with participants required to play one of the two roles, before then ranking a range of product ideas that had already been ranked by consumers.

As in the circus experiment, creators were much more likely to agree with the consumers when it came to ranking the most creative ideas.

Given the apparent poor performance of managers in evaluating innovation, it might be prudent for companies to reassess how the ideas generated by employees and other stakeholders are evaluated.

A form of peer evaluation may be considerably more reliable than management based assessment, or even to ensure that managers actually do a bit of the creating themselves so they engage that part of their brain too.


6 thoughts on “Research reveals that managers suck at judging new ideas

  1. The article draws much too broad a conclusion from the evidence.

    In the study, the question was how much an audience would like a circus act. The performers made better predictions than the managers. Your conclusion is that managers are poor judges of innovation.

    Judging the quality of a proposal for an innovation involves much more than just how popular that innovation will be; it involves degree of risk, strategic fit, cost, portfolio fit, time to market and any number of other political, technical, strategic and other considerations. These can all be assessed much better by a manager than by a front-line producer.

    “Research reveals that managers suck at judging new ideas” is a totally unjustified title for the piece. “Research reveals that performers understand audiences better than their managers” would be have been more honest – but that headline, of course, is hardly surprising and therefore less likely to attract clicks (including mine).

    • Thanks for your comment Graham. You're right, a lot more studies would be required to carry the inference across to other sectors. but I thought it would nonetheless act as a useful reminder that the people with the power to judge aren't often the best people to do that judging.

      • I think you are both right. The evaluation should be done in 2 steps, the first one being to identifie the best ideas (that's where the peers come in), the second one to asses whether the idea is worth developping businesswise (that's where the managers come in).
        Combining these two approaches would allow to make the best choise in regards to your custumers appreciation and avoid to invest in projects that will not be beneficial.

Leave a Reply

Your email address will not be published. Required fields are marked *