Discerning intent from tweets?

November 29, 2012

I have been quite busy of late with travel, proposal writing, and working collaboratively on some joint offerings.  Hence the delay in posting.  As those who drop by to see if I've posted anything from time to time know I'm at best a “once every couple weeks” blogger.

Today is a quick one and was triggered by an email I received today from a very knowledgeable CE practitioner.  In fact I'm often humbled by the people I have the pleasure to meet and train during our course offerings as most if not all are deep experts in their own right… many with advanced degrees and amazing lists of achievements.

So the question that was raised was:

Can one infer intent or sentiment from the content and frequency of tweets/re-tweets?  Am interested in the science/mathematics of such things.  Guidance?

My response: You can find interesting things with word counting and content analysis but our position is that it breaks down when you start to make assumptions of meaning.  Sentiment often seeks to go into the deeper meaning of content and this is the fundamental fault of semantic analysis.  We acknowledge that the content could provide interesting information and some directional information on sentiment however it can also put you at risk of completely missing the real sentiment or create a sense that sentiment is in a direction that its not at all reflecting reality.  Its these false indications which are most troubling.

So shorter answer is that based on our research and understanding of research in the broader field is that no, you can't infer intent or sentiment with confidence using this approach.  Now higher frequencies or co-occurences of words may serve a good trigger to explore more deeply in an area which done well can get you deeper insights but our expertise suggests you need a different approach that leverages human judgement across large populations.

The relevance of the Siri / Twitter image is to acknowledge the value of fragmented content AND a technology to help us process the spoken word.  The danger however as my question response above points out is when we over-extend our trust in the ability to algorithmically extract meaning from such fragmented information without more appropriately drawing on human judgement and cognition.

Would be great to get some thoughts from others on this.  Comments?

Recent Posts

About the Cynefin Company

The Cynefin Company (formerly known as Cognitive Edge) was founded in 2005 by Dave Snowden. We believe in praxis and focus on building methods, tools and capability that apply the wisdom from Complex Adaptive Systems theory and other scientific disciplines in social systems. We are the world leader in developing management approaches (in society, government and industry) that empower organisations to absorb uncertainty, detect weak signals to enable sense-making in complex systems, act on the rich data, create resilience and, ultimately, thrive in a complex world.

Cognitive Edge Ltd. & Cognitive Edge Pte. trading as The Cynefin Company and The Cynefin Centre.


< Prev

Singapore to Cardiff

I had too much good champagne on the flight back last night.  Every now and ...

More posts

Next >

Cynefin: revised leadership table

I managed to get back to daily blogging in November, albeit with the odd retrospective ...

More posts

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram