The Economist lead with a story this week on what they regard as a crisis in the science world.  The crisis emanates from what they regard as poorly constructed studies that are largely conducted due to the increasing pressure on academics to publish papers.

The result is that an increasing number of academic findings struggle for reliability and credibility, because an increasing number cannot be replicated a second time.  If you can’t run the same experiment again and find the same outcome, how valid was the finding of the original study?

Over the past few years various researchers have made systematic attempts to replicate some of the more widely cited priming experiments. Many of these replications have failed. In April, for instance, a paper in PLoS ONE, a journal, reported that nine separate experiments had not managed to reproduce the results of a famous study from 1998 purporting to show that thinking about a professor before taking an intelligence test leads to a higher score than imagining a football hooligan.

The article cites a famous Amgen study that tried to replicate 53 studies that were at the time considered landmarks in the basic science of cancer.  The Amgen researchers often worked closely with the original academics to ensure the studies were conducted accurately, but even so, they were only able to replicate the same findings as the original study six times!

It got me thinking about the staple of business education – the case study.  If so many scientific studies seem beholden to the particular circumstances they were conducted under, how repeatable are business case studies?  If the case studies are based upon the sound management principles that their proponents believe then surely they would be, yet many of the case studies mentioned in Good to Great famously went on to be anything but good or great.

Surely a major point of learning via case study is that it provides you with the kind of insights that allow you to replicate the behaviours exhibited in the case study.  It’s a basic form of ‘if you do this > this will result’, albeit in a narrative form.  If you start looking at case studies as wholly applicable to that unique moment in time for a particular business then the crossover lessons become much less evident.
Case studies are particularly defunct in social business contexts.  Look below at some of the things one is supposed to learn from using case studies:

• Analytical skills
• Decision making skills
• Application
• Communication skills

Lets look at each in turn.

Analytical skills

The theory goes that by virtue of sifting through the quantitative and qualitative data provided by the case study, you improve your analytical skills.  That may be true, but how many case studies will provide you with the kind of metrics that really matter?  Knowing what to measure is arguably more important than how to measure.

Decision making skills

The theory goes that you analyse the data available and then practice making decisions based on that data.  Here the term garbage in, garbage out springs to mind.  Organisations are incredibly complex things.  The skills and talents of employees are likely to be unique to that company.  The financial situation is likely to be unique to that company, as is their market position and countless other factors that go into making a successful decision.  Making decisions based upon simplified information doesn’t seem to be the soundest strategy.

Application

This is when you get to apply all of the tools and models devised by noted academics down the years.  The same models that are largely based upon the same kind of flaws we see with case studies.  If you could seriously manage by model then the management profession probably wouldn’t be in the funk that it is.  People should be trained to think for themselves rather than applying identikit solutions to unique problems.

Communication skills

This is probably the cherry on top, as it bestows the power of management upon you.  You have made your decision, now you communicate that strategy to your minions, using your excellent powers of communication to persuade and enthrall.  Of course things seldom work like that, and evidence shows that involving people in decision making is far more effective at communicating strategy than cascading it.

I’m not completely opposed to case studies, in that the narrative form can provide an easily digestible way of looking at an issue, but the key really is to think about your own situation and apply a unique solution to it.  You’re not the same as other organisations, so your strategy shouldn’t be either.