Scaling in complex systems

October 1, 2017

I wrote a series of posts about scaling over three year ago which in part addressed some of the perversities of methods focused on accreditation revenue to which I referred yesterday. They were important posts and sections will be dumped in Scrivener over the next months as I get ready for an intense booking writing week in a remote cottage on Ynys Môn this January. Naturally I have done a lot more thinking since then but the fundamentals remain: you don’t scale a complex system by aggregation or imitation but by decomposition to an optimal level of granularity followed by recombination (possibly exaptive) in context: complexity is about how things connect far more than what the things are. I remember an argument about this in IBM days when I had finished working through a radical new organisational model in which the role of IBM would be to centre a network rather than own resource. It was rejected on the basis that scale was perceived in terms of size and number of reports, in turn linked to hierarchical and aggregative/reductionist ideology.

The so-called scaled frameworks in Agile made, and continue to make similar mistakes with varying degrees of absurdity. The worst simply aggregates everything that has ever been called Agile into a complicated flow diagram, reducing complexity to an over-constrained structure. This is scaling by dumbing down to a lowest common denominator approach; it gives an impression of order by constraining practice to the explicit. As such it mimics the errors made in BPR which were magnified in Sick Stigma to the point of doctrine. The net result of all reductionist approaches to process management was the emergence of high energy wastage and stress as humans worked around the system to make it work despite itself; effectiveness was sacrificed on the altar of superficial efficiency. Interestingly in twitter encounters with some advocates of the SAFe Borg yesterday this (the idea that you work around something to make it work) was put forward as a defence – good consultants/coaches cherry pick what they think will work and modify it in practice while maintaining the pretence of the over-simplification they think is necessary to sell to senior IT Management. I think the phrase here is trying to force a virtue out of a vice.

To illustrate the point let’s take organisational forms. We know that the behavioural characteristics of teams of five, fiveteen and one hundred and fifty are very different. You can’t take the practices and interdependencies of a small team and replicate them in a large team. In a small team a level of intimacy and knowledge of the characteristics of other team members allows smooth coupling and pre-emptive actions to mitigate faults and weakness. In a large team we become increasingly dependent on the explicit statements of qualifications and competences and less on the tacit understanding of capability in different contexts; compliance and conformity are privileged over anticipation and adaption. Taking a complex perspective we would create variable expectations of different identity clusters (people can belong to different identities) with management and monitoring of the nature of interactions and emergent stabilities that arise from those interactions. This allows a large system to grow very quickly and at times in novel and unexpected ways as a series of situated networks. Think of a Portuguese Man of War which is a symbiosis of different creatures which assemble in a specific context, or of Slime Moulds which again change their nature under conditions of starvation; in software Object Orientation and Micro-services allow novel combinations to emerge if properly architected. It’s not that we don’t have the models to use here, its just that we are stuck in engineering diagrams.

Alicia Juarrrero famously coined the phrase like bramble bushes in a thicket to describe a complex adaptive system and it is a title Kurtz and I used in this paper which represents early thinking on organisational design and networks. In a complex system you need to define your identity focal points and manage your connections, rather then get out the flow charting template and create excessive structure. Bramble bushes bear fruit, but also prick the unwary; structured flow diagrams are safe in the sense of the absence of novelty, the absence of risk and the absence of imagination.

4 responses to “Scaling in complex systems”

  1. David J C Morris says:

    Provocative article, as always Dave.

    Firstly, let me say that I am a practitioner not a consultant. I do not get paid to train people nor to configure processes and systems.

    I am a pragilist. I like what works, and what works differs for different contexts. While these days that is mostly some flavour of lean or agile; as you would expect from someone trained by you in Cynefin, there are other models that work better for other contexts.

    Sometimes what is appropriate is pockets of individual teams doing their own thing on products that don’t need to integrate or that manage integration through a series of micro-services and contracts. Sometimes it is a network of disparate teams who interface on as as-needed basis and can be trusted to coordinate and synchronize accordingly. Sometimes it is a multi-tiered model of teams within teams within teams.

    I like Nexus, I like LeSS, and I like SAFe. I have never succumbed to the temptation of saying that I had a favourite uncle or that I loved one child more than another. That’s the land of disharmony and false dychotemy. I choose not to live there.

    Underlying many such arguments against SAFe seems to be a thread that says; because SAFe is a framework of disparate practices, there is no guarantee that it will work. That is true. That is accepted by most SAFe practitioners. That is also built in. It is a framework. It is a starting point.

    While SAFe offers some metaphors to help management understand, it is based on sound agile principles and practices. Chief amongst them being transparency, inspection, and adaptation. Of the framework as well as the product.

    Even Scrum, that holy of holies to anti-SAFers, calls out that we can adapt Scrum once we’ve run with it for a while.

    And here’s what it boils down to. Most times I hear this argument, it is from people with competing frameworks. It is negative marketing.

    Ironically, it is what we used to do to distinguish agile from waterfall practices. Me included. Guilty as charged. Mia Culpa. Mia Maxima Culpa. But then I learned. From deeper understanding of the agile manifesto. From reading more broadly. From other practitioners. From meditation. And, from yourself too, Dave.

    Critically, it is against the very first value pair in the agile manifesto. It places adherence for or against a process above the respect and willingness to listen and collaborate with other practitioners.

    I expect that from Ken. I have said the same to him too. He has been negative about Dean and SAFe for years. Mostly because he was pushing what grew into Nexus. It’s not forgiveable, but it is expected.

    So it was not surprising to see that you now also have products in development. Disappointing, but not surprising.

    Perhaps, these days, we just do whatever it takes to generate an income. Perhaps I have misunderstood the intent and the action? Either way, as a pragilist, I await them with interest, of course.

    However, I don’t like the way discourse is going. It’s creating Trump-isms in the agile world, and it’s ugly. Could you please stop killing my new-age fluffy bunnies?

    From a fan (still).

    • Dave Snowden says:

      Sorry, but the idea that anything is good and everything should be accepted is not something i agree with. SAFe basically is trying to sweep anything remotely Agile into a training and certification framework. It started selling by using a pyramid selling scheme with no requirement for experience. Now it is established it is cynically trying to raise the cost of entry (and thereby its revenue). I constantly tell people that I am sure that good consultants can make the monstrosity work despite itself, but the overall construct is ontologically incompatible with Complexity.

      Now the classic response (and you use it to a degree) is to say that Scrum is just as bad, or that Ken just wants to make money. This is deeply cynical and wrong. Scrum is a technique not an over arching framework. Nexus looks (but I am still looking at it) as consistent with CAS and LeSS at least works incrementally. I think that Ken, like me, thinks SAFe is a betrayal of Agile and neither of us would seek forgiveness, or accept your attempt to explain that away as a rival commercial interest.

      When SAFe added in DevOps the cynicism was clear

      As to our offerings – we run training in a variety of things, we offer no certificates, no accreditation. We are under pressure to provide some type of quality control and I am looking at ways to do that. Anything we do will include an apprentice period under supervision and will not be linked to a training course. If we do anything it will relate to specific methods and tools not a whole field. You have very much mistaken the intent and action.

      SAFe is not a fluffy bunny, it is a highly structured near waterfall fox hiding in a rabbit skin

  2. Luca Minudel says:

    So far I’ve found two interesting ideas on scaling complex systems:
    – patches from Kauffman
    – and this paper on how to take into account the local social context
    ‘Change in complex adaptive systems : a review of concepts, theory and
    approaches for tackling ‘wicked’ problems in achieving sustainable rural
    water services’

    Which authors and books/articles are you considering when you look at the theory and practice of scaling complex socio-technical systems?


Leave a Reply

Your email address will not be published. Required fields are marked *

Recent Posts

About the Cynefin Company

The Cynefin Company (formerly known as Cognitive Edge) was founded in 2005 by Dave Snowden. We believe in praxis and focus on building methods, tools and capability that apply the wisdom from Complex Adaptive Systems theory and other scientific disciplines in social systems. We are the world leader in developing management approaches (in society, government and industry) that empower organisations to absorb uncertainty, detect weak signals to enable sense-making in complex systems, act on the rich data, create resilience and, ultimately, thrive in a complex world.

Cognitive Edge Ltd. & Cognitive Edge Pte. trading as The Cynefin Company and The Cynefin Centre.


< Prev

The certification question

The vexed question of certification came up in conversation yesterday. We’re going through the process ...

More posts

Next >

In the Tricotocon …

Over the years I’ve developed a range of facilitation and workshop techniques based on the ...

More posts

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram