Some concluding thoughts from Manchester Airport

So what does all this add up to in terms of the use of Cynefin and Sensemaker ideas in the evaluation field.

I think there is good news and bad news. Let me deal with the bad news first. There are four aspects :

Legitimacy. On the whole the evaluation field draws inspiration and legitimacy from applied social research and management planning. There are elements of community development and anthropology but they vary in importance. Evaluation has a history of innovation within the context of those fields but often has significant problems incorporating innovative ideas from other fields. And incorporating them well rather than superficially. Sensemaker in particular is way outside the boundary of most evaluation theory and practice and therefore is vulnerable to being treated either with skepticism or incomprehension. Evaluators know so little about the debates and developments in the knowledge management field that it is easy to dismiss the validity underpinning the ideas and practice of Sensemaker. In North America especially, the evaluation field is very guru led – often university based, with great access to the resources of these institutions including doting students to pass the message on.

Resources. Evaluation is, in general, very resource limited, especially in terms of the sheer complexity of the situations it is seeking to gain insights. Budgets are tiny, timetables restricted. Consequently evaluation tends to favour cheap fast methods without a great deal of upfront investment, either in terms of cost of skill development or financial cost.

Usefulness. Recall that evaluators have little use for information that doesn’t tell them about what is worthwhile about an intervention. If that is not obvious from the application of a method then it is likely to be dismissed as a poor investment of time and money.

Stakeholder relationships. Evaluation has inherited from the applied social research and planning fields the tradition of the expert written report and the notion of “independence”. Those who commission evaluations are sometimes reluctant to engage in deliberation with evaluators of emerging results that may violate their notion of independence.

Sensemaker fits uneasily within these four generalisations for reasons that are fairly clear. The exceptions to these generalisations of course provide opportunities for Sensemaker, although the Cognitive Edge network would need to have deep roots in the evaluation field to spot and exploit them. Sensemaker and Cynefin have yet to find a major champion with the necessary authority to legitimise the concepts. While Irene and I are well known in the evaluation field we do not have the kind of clout I am talking about.

And now the good news.

The good news is that some parts of the evaluation endeavour is in a state of crisis. In many cases evaluators being asked to address complex aspects of situations with tools that are primarily appropriate for dealing with complicated and simple aspects of those situations. Many standard social research methods and management planning methods are at best inadequate and at worse completely invalid. There are very few tools in the evaluation marketplace that has the potential of Cynefin and Sensemaker – maybe some from the fuzzy logic field, the Max Boisot end of the strategy field and from some aspects of network analysis. Evaluators are also increasingly aware of ideas of complexity even if the methods they currently use to engage with it are often lacking in necessary power. That’s the Cognitive Edge network’s major opportunity, but it has to address the bad news – and as someone promoting Cynfin and Sensemaker within projects that’s not an easy task. Hopefully the work that David and Irene are putting in will help address some of these issues and I’m looking forward to their contributions.

Many thanks for staying with me on these two week’s of blogs. In that time I’ve travelled half way around the world, facilitated a five day meeting of evaluators wresting with just these issue, tried to help colleagues make sense of some Sensemaker work and participated in some challenging conversations about how to describe the logic of interventions… and where a complexity frame might fit. Those conversations have informed the blogs in a way that I hope helps to move the Cognitive Edge mission forward. See you in an evaluation some day.