The poverty of surveys

January 26, 2008

Ever filled out an employee satisfaction survey? I’ve lost count of the time I have wasted on these over the years, both in completion and interpretation. The questions asked contain a hypothesis as to your motivation. Does your manager consult you on a regular basis? to which a response ranging from all the time, to not at all is expected. My response was always along the lines of it depends. However the subtleties of context were not allowed and I was told to answer it on average or overall, to my mind such an instruction invalidates the input. Interpretation was worse: Your division shows a 5% decrease in employees who are satisfied with their overall terms and conditions, but 10% more feel that they are treated fairly (that is a real case). What is one meant to make of that? Overall too many assumptions are built into the questions and the results provide numbers without context and this the lack meaning. Fortunately there are now alternatives, but more of that later.

At lease the satisfaction survey is, in general well motivated. Employers do want to know what is happening amongst their employees. However there is another, and scurrilous use of the survey instrument, namely the supposedly objective facts disguising a sales pitch. A good example of this can be found here in Melcrum’s survey of employee engagement, linked to the publication of their Practitioner’s Guide to Employee Engagement .

I have extracted the following quote from the survey: The survey revealed that 86% of organizations with employee engagement on the agenda are using at least one of the four most widely used techniques – action teams, Appreciative Inquiry, message maps and storytelling. Now I would like to know on what basis that claim is made, it does not appear that this conclusion arose from unprompted responses to the survey. It does look as though respondents were asked about those four items specifically, and surprise, surprise those just happen to be four chapters of the guide.

Story Telling it turns out is in use on 49% of the survey subjects. Now the naive and over enthusiastic reader might take that as an endorsement of the subject, but is it really? The survey is based on 1,625 responses from professionals working primarily in communication or HR. So if anything it is surprising the response on story telling was not a lot higher. I can’t imagine anyone in communication not answering yes to their use of story telling so if the split is 50-50 between communications and HR then its bad news for the technique in HR departments. In any event only 27% of the respondents even have an employee engagement programme in the first place.

Mind you, given that story-telling and appreciative inquiry are techniques designed to achieve management objectives, not to allow the authentic voice of the employees to to be heard I can sort of believe that the figures may be true. Having executives tell their employees a story has always seen to be an very impoverished use of narrative, better, and more honestly described as propaganda. Requiring your employees to tell positive stories (Appreciative Inquiry), refusing them the right to tell negative stories represents another Orwellian intervention.

Generally these type of handbooks are a process of dumbing down anyway. One of the authors, a good friend of mine Tony Quinlan, on his blog bewails the process of simplification that he was required to go through. It looks like he took the position that it was better to write the article with qualifications, rather than let a platitudinous charlatan take up the pen, and I happily support him in that. He makes a good reference to the Mullah Nasrudin story of the Falcon to illustrate his point.

The ultimate condemnation here is the price. Melcrum want £225 for four articles and a survey. Now most of the material in those articles (if not all) can be obtained from the wikipedia and various articles available on the internet. If you want to go beyond that then four books covering the four subjects would cost you less than £100. This is a con trick. Regrettably most of the publishers are going down this route. Ark (who should know better) are allowing Uncle Jerry to publish the contents of an open, free access list serve at a similar price!

There really are better ways to do this. Narrative work in organisations is about understanding and extracting meaning from naturally told anecdotes (the ones they want to tell, not the positive ones demanded by the appreciate inquiry process) that allows people to make decisions. Numbers with context come from narrative based research, and that enables action. Complexity techniques such as social network stimulation (we teach that on the Cardiff course along with narrative) allow employees to engage with colleagues across silos and outside the formal boundaries of the organisation.

One day, sanity may break through ….

Leave a Reply

Your email address will not be published. Required fields are marked *

Recent Posts

About the Cynefin Company

The Cynefin Company (formerly known as Cognitive Edge) was founded in 2005 by Dave Snowden. We believe in praxis and focus on building methods, tools and capability that apply the wisdom from Complex Adaptive Systems theory and other scientific disciplines in social systems. We are the world leader in developing management approaches (in society, government and industry) that empower organisations to absorb uncertainty, detect weak signals to enable sense-making in complex systems, act on the rich data, create resilience and, ultimately, thrive in a complex world.

Cognitive Edge Ltd. & Cognitive Edge Pte. trading as The Cynefin Company and The Cynefin Centre.


< Prev

Consultations as part of perverse power

The government is consulting about the section 60 legislation under the Health Professions Order 2001 ...

More posts

Next >

Confusing symptoms with cause

An interesting set of presentations down here in Tampa, Florida over the weekend. More reflections ...

More posts

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram