Evaluating SenseMaker

As I mentioned, 2011 is my year of action research on SenseMaker. I’ve got six cases to draw on (various stages of implementation), more than enough for me at the moment.

I’ve set myself the following questions (thanks to Rick Davies for input) that I want to answer at the end of the year, focusing on what SenseMaker contributes to evaluative practice in international development.

1.When and where is this approach useful?
•In which contexts has it been useful and how
•What are pros and cons? And its USP?
•In particular, how can it contribute to learning and help in being accountable (whether through an attribution or contribution logic)?

2.How do you assess the quality of results with this method? Not results of the work you’re doing but the findings generated with SenseMaker.
•How do we need to understand rigour in this approach, how is it embedded?
•Where are potential slippages in quality (story collection, understanding of the unique triads, data entry from paper, analysis and inference…) and how to spot and correct for these?
•What is needed to meet the full set of evaluation standards: utility, feasibility, propriety, accuracy and accountability.

3.How does this approach compare to other approaches? In particular,
•Are its proprietary arrangements a possible bottleneck? And how, in fact, does it relate to the notion of public vs privatised knowledge and methodology?
•What are the merits and limitations (operational, analytical) of its focus on mass capture, high volume of fragments?

4.Where does it challenge mainstream M&E (monitoring and evaluation) conventions, such as on issues of linear planning by suggesting we look at dispositionality and adaptive engagement instead? And is this a convincing challenge?

5.How does it deal with the complex domain, as per the Cynefin heuristic?

Any other ideas of questions to think about?