Tools for too much text

September 13, 2010

One of the hazards of doing research with a broad focus is getting lost in all the information out there. One of the pleasures is just wandering around the internet looking for stuff. Data overload is one of the main reasons I was so grateful to come across the Cynefin framework in late 2006. Apart from this leading later to the opportunity to use SenseMaker™ as research tool, conversations and stories told in doing the accreditation course in Brisbane also led to some exposure to other ways researchers, governments and organisations were using to find important information in big volumes of data.

In 2008 I had my own close encounter with the detailed analysis of text using transcription and NVivo. Learning first-hand how intense and time consuming this approach is drove me to not only make sure I had the opportunity to use SenseMaker™, but also to keep my eye open for other options. It’s not that the results of detailed analysis weren’t valuable, just that the price seemed far too high. I started out looking at semantic analysis and word density tools and stumbled into Latent Semantic Analysis, anti-plagarism software Turnitin and iThenticate and text analytics and I’m still looking.

Plagarism is a big concern in academia and even a simple search on Google will turn up many things that have clearly come from the same place, attributed or not attributed. Churning out words can turn into more than an embarassing slip-up for a professional writer . Stories can echo around the place. I get the feeling that often people are either just too busy or don’t have the skills with the nuances of language to create an original response to something read or heard, it’s easier to just repeat it on just like telegraph did in the 19th century.

Data mining is huge and if you don’t have the technical background it seems a bit like divining for water. It does help if you know the water is there already. Text analytics have been developed for academic purposes and more general use. Examples include tools such as Leximancer and quintura though I am sure there are more (please feel free to comment on any good ones you know about). This post just marks some thoughts on how the options of finding ideas in large volumes of information are evolving.

I was reading today about how Benjamin Franklin received a ‘could improve’ note from his teacher as a young child and was encouraged to teach himself how to write eloquently by reverse engineering a much admired text.
Maybe part of the challenge and part of the solution is that the text and stories that link together big ideas use prose that is well beyond the day to day language people have learned to use. The best explanations make clever use of metaphors and myths to get complex ideas across to others. Part of me does not trust that the hocus pocus of semantics can really find these true gems, many of which I have come across purely through speaking with people, serendipity, rumours and persistence. Which is why I like story and narrative and using naturalistic sensemaking and SenseMaker™.

Recent Posts

About the Cynefin Company

The Cynefin Company (formerly known as Cognitive Edge) was founded in 2005 by Dave Snowden. We believe in praxis and focus on building methods, tools and capability that apply the wisdom from Complex Adaptive Systems theory and other scientific disciplines in social systems. We are the world leader in developing management approaches (in society, government and industry) that empower organisations to absorb uncertainty, detect weak signals to enable sense-making in complex systems, act on the rich data, create resilience and, ultimately, thrive in a complex world.

Cognitive Edge Ltd. & Cognitive Edge Pte. trading as The Cynefin Company and The Cynefin Centre.


< Prev

Seeing perspectives

A big thanks to David Williams who has been the guest blogger since 30th August. ...

More posts

Next >

Stretching the definition of narrative

I'd like to explore the idea of narrative not being limited to words or text. ...

More posts

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram