It is never that simple

If you read the report I referenced in yesterday’s post you will find multiple claims that there is growing evidence that mindfulness has a major impact on health and well being. By coincidence I came across a New York Times article this morning reporting that “a team of filmmakers led by a British cardiologist say that the function of the Mediterranean diet may have been oversimplified. They contend in a new film that the region’s good health is driven not only by food, but by an array of lifestyle factors, some of which they claim have been overlooked“. Now this may or may not be accurate but it points out the danger of confusing correlation with causation. Also the related danger of confusing symptoms with causes.

Both of these confusions are all to common in in the fields of evidence based policy and snake oil purveyance consultancy methods, and we can add a third issue namely the danger of biasing the results based on commissioning expectations. As an aside here, I have long thought that the absence of debating from schools and universities is one of the factors which has generally reduced the capacity of people to exercise critical reason. Not just performing by the way, but listening. My pet rant over lets look at the three issues.

1. Confusing correlation with causation may itself result from physics envy, but it is a real problem in management science.  Even satire can’t debunk this issue and there is some evidence that is part of what we are as a species and is more likely when aspects are presented in a distinctive or unusual way; a feature all too common in presenting the latest magic pill.  Aside from an illusory correlation it may well be that thing cause and effect themselves arise from some other factor.  So taking up meditation improves someone’s mental health; but it may be that the decision to do something is more important  than the thing chose.  So walking, cycling or changing social interaction could have had an equal impact.  In fact it may be many things in combination.   
2. The symptoms and cause confusion has always been more dangerous in method development.   I have long used the example of confusing creativity with innovation; the former is a symptom of the latter.  But if we focus on an observational/deductive model of research its a danger.  All these people have been innovation, they are also creative so if we make people creative they will be innovative.   Healthy people meditate, so if we meditate we will be healthy and so on.  Life really is (as the title of this blog suggests) that simple.  Multiple causes or rather dispositional states with varying modulators are key to understanding complex systems no linear models of single point causality.
3. My third danger is more perfidious and impacts on a lot of commissioned reports.  So a government department is looking at mindfulness to deal with more motivational factors  and they commission research to see if there is an evidence base.  The organisation awarded the contract already knows what the goal is and they focus their search on material that links mindfulness to motivation which means a partial selection from the field and a reading which is predisposed to a conclusion.  Especially as these days in research the junior staff may have less that five teen minutes to skim each article in a search and draw conclusions.   Evidence based policy has become an industry in its own right, more policy based evidence these days.

In a multi-causal system we need to run multiple parallel interventions which have some basis in science (where we can do repeated tests of hypotheses). This is the test for coherence, then patterns or dispositions of behaviour will start to emerge which may or may not shift things in the right direction in a specific context. Evidence itself is an emergent property in a complex system. The only magic pills available have a correlation with cash not cure ….