Extispicium

The Sloan Managment Review recently published a paper on evidence based decision making which is well summarised in the abstract as follows: Many managers think they’ve committed their organizations to evidence-based decision making – but have instead, without realizing it, committed to decision-based evidence making. Is that all bad? What can be done to fix it? Its a good article has been picking up a fair amount of traffic in the twitterverse.

I’ve been concerned for some time by the growing body of material that seeks to enforce a “fact-based” approach to decision making and its nice to see an article picking up on this, and some of the holy books of that movement such as Competing on Analytics by Davenport and Harris. Now we need to be careful here as the management pendulum tends to swing from process to intuition with alarming frequency; to my mind the real problem here is attempting a universal solution rather than one which is contextually bound. Of course the harsh reality (as the article points out) is that in most cases evidence is assembled to support an existing or anticipated decision. One can do wonders with a spreadsheet and a statistical package to meet the requirements of power.

Protesting the phenomena i a waste of time. If I remember by Roman history aright, the Emperor Claudius was an early offender, and even if I am wrong in that fact I am not wrong in saying that this issue has been around for a very long period of time and it is not going to go away.

So what do we do about it? Well the authors of the article make a valuable point when they say: The difficulty that arises in practice is that organizations may have trouble differentiating between situations in which disconfirming evidence should be heeded and situations in which disconfirming evidence can safely be dismissed. I am less sure of the value of their advice, although it is par for the course in articles of this nature. Roughly summarised their recommendations are:

1. Decide if the decision problem can be supported by formal evidence and have the courage to make a decision based on intuition if it is not.
2. Determine if it’s cost effective to gather the evidence in the first place
3. Be aware that evidence may be needed for external audiences as it provides significant ceremonial and signaling value, internal audiences are less easily fooled.
4. Ensure than objective evidence if reflected more often than not in decisions, feed manufactured evidence to internal audiences rarely and sparingly.

Now I read these with some wry amusement and admiration for the cynical realism that three and four in particular represent! The first two are sound, but are pretty self evident, no attempt is made to provide the how, and without that provision any such statement is a platitude. I have two main objections/concerns here:

1. It’s a grave mistake to talk about intuition as an alternative to evidence, aside from the fact that its one of those dangerous and seductive dichotomies it also creates an either/or when it should be a both/and.
2. While it acknowledges that different contexts may exist, it defines those contexts by the solution, not by their nature. This makes it more difficult to put into action.

As it happens I have been talking about evidence based decision making on the seminar programme here in Australasia for the last fortnight. An essential aspect of that has been to distinguish between decision making in the complex and complicated domains of the Cynefin framework. This involves understanding the nature of evidence in the context of the system. I’ve taken each group through a basic method to look at a problem from different perspectives which you can read here. Basically it says uses a wider group than the leader to determine the nature of the system and then takes two views of evidence:

1. If the issue is complicated then there should be a body of evidence that (assuming its cost effective to get it) will allow you to determine the right answer.
2. If it’s complex then the facts will support multiple theories about what to do. Evidence then does not relate to a predictable outcome, but rather to a determination of the coherence of the theory and proposed practice of any and all ideas about what to do next. If the proposed intervention is coherent to the facts (and several contradictory interventions will be) then a safe-fail intervention is justified by the evidence, provided that it can be monitored and critically amplification and damping strategies for success and failure are provisionally sketched out in advance.

Now there is more to it than that, and the method document provides some direction. As an approach it recognizes the importance of context in decision making, the danger of universals and the poverty of platitudes.