Patterns & significance

A series of emails and skype calls today built up a head of frustration with some people’s obsession with statistics and data. An obsession that attempts a substitution for sense-making, rather than as a tool to support it.   One conversation with a research head of a British local authority who had been passed a HBR article which said data was more accurate than human judgement in recruitment and sooner or later all recruitment would be a big data algorithm.   Another with a colleague frustrated by people talking about the validity of statistical patterns for low volume datasets and not realising that simple visualisation can enable an intelligent person who make sufficient sense to act.  Then there were a series of calls wondering what number made a data set valid, the question seeking some absolute statement independent of context.

There are a few things we are missing here in particular the question of human judgement.  We need a conceptual reset, a wider perspective: indeed a total perspective vortex is an appealing location for some people.  nbsp;So let me put forward some simple statements, just short of a round dozen:

1. The purpose of statistics is to inform human decision making
2. Correlation is too often confused with causation as simulation is confused with prediction.
3. Humans can change the context through intent (or accidentally) which can render any statistics out of date/data/meaning
4. Taleb’s best book ​Fooled by Randomness has multiple examples of the danger of removing human judgement from the equation
5. A weak correlation maybe an indication of an abductive linkage, a weak signal that needs human intelligence to interpret.  At all costs in a complex system do not abandon outliers
6. Decision making is about a sufficiency of information and a coherence of hypothesis to determine a way (or ways) forward; we do not and cannot require certainty
7. The number of observations needed is determined by (i) statistical theory and (ii) the credibility of your results.   The more difficult to accept the more data you need and the fact this is the case, makes a wider point about limitations
8. In SenseMaker® if we present a colour coded dot map on a triangle, humans are more than capable of seeing patterns and using the micro-narratives to glean context.  It is about sufficiency and caution rather than the incautious use of false certainty
9. Measuring flow in a complex system is more important than a statistically accurate retrospective reporting.  Presenting real time data to decision makers which show changes (including ones which may not be statistically significant yet) over time is far more useful
10. Abrogating human judgement is to encourage evolution to either (i) cull out the capability for judgement from future generations and/or (ii) cull those stupid enough to take it seriously.   I hope for the latter, I fear the former.
11. Don’t get seduced by individual judgement, seek out the wisdom of a human sensor network not a crowd.

I could add more but that is enough to be going on with.  And with the theme of judgement a warning about fate in the opening picture, although I reserve the right to be more reasonable under questioning.