Differentiation vs Integration

Read an interesting post today by Richard Veryard looking at the current state of business architecture.  In particular Richard was musing on the traditional dichotomy between standardisation and integration vs differentiation and autonomy from a business perspective.  Essentially Richard was using the Clive Finkelstein  adapted version of the two-by-two matrix from Ross, Weill and Richardson’s “Enterprise Architecture as Strategy”, below.

 

 

Firstly Richard points out that the general tendency is for everyone to think that ‘unification’ (so highly standardised, highly integrated capabilities) is the best place to be.  I’d first like to say that this is rubbish.  The reality is that an organisation – whether they know it or not – consists of many different business capabilities and the best position depends entirely on the capability in question. 

To start to break this down against the two axes, lets broadly examine integration and then standardisation:

• If we consider a typical enterprise to increasingly be a collection of business capabilities that reside inside and outside the boundaries of a single firm then we can see that integration is indeed becoming far more important.  Integration in this context, however, does not require the traditional tight integration of an end to end business process but rather the loose integration of autonomous service providers that collaborate to deliver value.  In this sense whilst ‘value stream’ integration may be high, ‘business process’ integration needs to be kept deliberately low;
• On the other hand differentiation of business capabilities is also claimed to be becoming more important as a result of increasing commoditisation and everyone is trying to understand how to be more agile in their offerings, more responsive to their customers and more relevant to their local markets.  Agility often leads to lower levels of integration and standardisation, however, as smaller groups seek the autonomy necessary to optimise “their” part of the business as it operates in the context of their location and stakeholders.

As a result the simple answer would seem from a business side demand perspective that ‘diversification’ (as requiring low business process integration and being potentially the greatest enabler of agility) is the new top right whereas most IT people are still aiming for ‘unification’. How do we resolve this seeming paradox?

Business Process (dis)Integration

The first issue to tackle is the seeming requirement of ‘business process integration’ as an indicator of increasing maturity in the graph.  Such language was potentially appropriate when business processes were seen to be the highest level of abstraction available to a business architect and when internal departments were the only major source of supply.  In this context it was common to try and eliminate waste by “hardcoding” all of the end-to-end process steps required to realise some value for the customer.  In this practice you  would usually have one or more architects who worked out the whole detail of how a process worked end-to-end in terms of who did what, how they did it and when they did it by – Taylorism was rife.  Such practices – whilst still widespread – are now essentially bankrupt at the macro level as a result of new methods and the rise of the web.

Business Capabilities and Extended Value Streams

I’ve talked many times about the use of business capabilities to document the stable structure of an organisation and provide a framework for concentrating on ‘what’ the organisation does rather than getting lost in the mess of ‘how’ it actually does it.  Business capabilities allow us to concentrate on the outcomes we require and map value streams both within and across organisations to coordinate these outcomes but ignore how the value is delivered.  This allows us to federate out the issues of implementation to the owners of each capability and give them the maximum scope for optimising their capability.  This is particularly important when you realise that codification of outcomes gives you options for supply – given that transaction costs have now made use of external capabilities highly desirable – but also requires you to let go of tight control of implementation and trust your chosen partner.  I wrote a long post on this whole subject here for those who are interested.

As a result whilst it might seem pedantic to separate ‘business process’ integration from ‘value stream’ integration I strongly believe that this extra level is needed to differentiate between tight integration of how you do things versus coordination (and hence loose integration) of autonomous outcomes.  This point is highly important in the context of our discussion as it allows us to understand that it is a) possible to identify and decouple the different business capabilities that we require and b) optimise these capabilities in different ways based on their characteristics.   This latter point is crucial as we look at the issue of whether the above matrix is sufficient to categorise emerging models.

No Capability is Created Equal

The first thing we have to realise is that the different business capabilities that implicitly make up an organisation today will respond to different kinds of strategies and thus require different cultural and economic thinking.  Once we have taken the first step and used business architecture to identify these discrete business capabilities we can begin to use this insight to better understand the optimum business architecture of each.

Different ‘business’ types within an Enterprise

Broadly there are four different kinds of businesses emerging as a result of the rise of the web and plummeting transaction costs.

• Infrastructure businesses: These businesses are essentially business capabilities that respond to economies of scale.  In this sense they need to be highly standardised and have tight integration within their internals – to drive low cost and reliability – but won’t be very innovative or have time for the deep relationships required to meet the needs of individual customers.  Examples of these businesses could be literal platforms – such as a telephony network – or ‘standardised’ services such as HR.  From an economic perspective such business capabilities will be driven to providing services at the lowest possible price (and thus the highest possible scale) but they will also provide relatively stable returns given the broad consumer groups that would leverage them and the difficulties that new entrants would have due to the capital requirements.  It is worth saying that whilst this may look like commoditisation – and thus something to be feared – in this business type the embracing of commoditisation whilst maintaining an ability to operate profitably is actually your differentiation.  From a cultural perspective the business model would require an organisation that is highly standardised, focused and repetitive;
• Relationship businesses:  These businesses are essentially business capabilities that leverage deep relationships.  Basically relationship-based capabilities rely on deep knowledge of their customers and the market they operate in to bring relevant products and services to their attention at the right time.  These capabilities are less able to be standardised – as they rely on personal customer context – and are also not likely to be strongly innovative as they will generally be responding to the stated needs of their customers – who only know what they know.  Examples of such businesses could be general practitioners or financial advisors.  From an economic perspective these businesses can take advantage of economies of scope – as they focus more on their target customers than on specific product or service niches – and can be very profitable due to the value they are perceived to add.  Culturally the business model would need to be highly flexible to build strong networks and to deliver combinations of products and services that were specifically suited to their target groups;
• Innovation and commercialisation businesses: These businesses are essentially small, innovation focused capabilities who specialise in IP generation and product and service commercialisation. They will often be one step removed from the demands of current customer need in order to have the time away from delivery pressures and thereby work on breakthrough products and services. Examples of these kinds of companies might be software startups or pharmaceutical research companies.  Their economics is based on intangibles and a long term growth in capital worth from the leverage of their successfully commercialised IP.  Their culture will be very loose, collegiate and focused on research and exploration and many of their endeavours will not result in any tangible success – as a result whilst the rewards for the small number of successful projects are high there is also the fact that most exploration will result in no valuable IP; and
• Portfolio businesses:  These businesses represent the residual strategy and investment capabilities of large companies and will be run as a separate business type. They will continue to own and manage brands but will replace direct control of delivery capabilities and assets with investment in a portfolio of specialised organisations. Examples of this kind of business might be Virgin or a large IT company such as Fujitsu or IBM.  Their economics will be primarily risk-based, investing money in a range of different organisations across categories in order to leverage their brand, balance their portfolio and maximise their return on capital. As a result they will invest in relationship businesses, infrastructure businesses and innovation businesses in order to get the right mix of stable, low growth returns and high risk, high growth returns.

Differentiation is…. different

Each of these categories of capability needs to maximise its assets in different ways, leading to completely different economics and cultures. I believe that these pressures – coupled with the decreasing transaction costs of working with others – will drive the fragmentation of organisations along these fault lines, since trying to manage them within a single organisational context will necessarily emphasise one whilst sub-optimising the others. Even without this, however it is plain that the different capabilities required to realise an end-to-end value stream need to be managed and optimised in different ways – with management styles, measures and dashboards altered accordingly for each – if we are not to sub-optimise the whole.  As an example, given that infrastructural capabilities are the largest (and often easiest to manage due to their concrete nature), many organisations tend towards management styles and metrics that reflect a concentration on these capabilities; concentrating on efficiency (in order to optimise your infrastructural capabilities), however, will destroy your ability to build deep relationships or to have the detachment necessary to innovate or invest successfully. As a result those organisations which manage to separate these different concerns and specialise – and in so doing create the right culture and economic metrics to underpin appropriate behaviours – will see much greater success than those who continue to operate their organisation with muddled thinking.

There is no Paradox, only Greater Subtlety… and Opportunities

The major implication of these insights is that there is no single ‘strategy’ box that an individual enterprise can be allocated to.  As Richard states in his post (my emphasis):

“….a two-by-two matrix would be misleading. The point isn’t to choose whether you have differentiation and integration or not; the point is to determine how much and what kinds of differentiation  and integration you need.”

I would also add to this understanding where you need that differentiation or integration (i.e. within which capabilities).

If each business is viewed as the portfolio of internal and external capabilities required to realise a given set of value streams then we can envisage a scenario in which each business strategy will be:

1. An overall statement of intent as captured by the target structure of the enterprise (i.e. its intended portfolio of capabilities, the outcomes they will need to deliver and the sourcing strategy for each); plus
2. The many federated strategies required by the individual capabilities to meet their outcome obligations. 

In this context tools like the matrix above may be helpful for enterprise strategists and capability owners to select appropriate strategies but only after they have evaluated the ‘type’ of capabilities they have and made prior strategic decisions about core competence (relationships, infrastructure, innovation or portfolio) and divested the implementation of the rest to specialised partners.  Those that remain can then be optimised as suggested based on their nature.  As immediate – but not very well thought through, lol – suggestions: infrastructural capabilities will tend towards ‘replication’ and ideally ‘unification’ strategies, relationship capabilities will tend towards ‘coordination’ strategies, whereas innovation and portfolio capabilities would likely tend towards ‘diversification’ strategies. 

As a result there is no paradox at all, only greater subtlety uncovered by increasing sophistication in the available body of thinking about business models and architecture.  The irony is that not is it only possible for a business to pursue both differentiation and standardisation strategies at the same time but rather that they increasingly must do so if they are to optimise their value streams and be competitive.  To do this, however, they must understand themselves as a portfolio of capabilities that need to be selected, retained or outsourced and then optimised based on their cultural and economic imperatives; unfortunately, however, not many organisations have even begun the process of working this out yet and thus business architects and business architecture are still more like honorary titles for valuable but difficult-to-place individuals than a widespread discipline.

Enterprise Architecture is yesterday’s news

Amazed to find this post that suggests that Enterprise Architecture is the next big thing.  I thought that EA was a late 90s, early 00s silver bullet and that we were now all moving on to a concentration on fractal business architectures supported by federated IT.  Now I realise that judicious EA can encompass this view – indeed that it should – but realistically EA as a term was hijacked by geeks who used it to enforce technical standards and who gibbered at their business colleagues in a threatening way about governance and compliance whilst creating impenetrable mountains of useless documentation that was out of date as soon as it was created (and made no sense to business people or developers). 

Worse than that most EA practitioners genuinely believe that they can document the whole organisation top down in a complex chain of dependencies – a horrible fallacy in a world of federation – and then micro-govern people by putting cumbersome processes in the way of getting anything done (and given the difficulty (nay – impossibility) of doing this they end up obsessing on forcing shared and inappropriate IT on people instead of supporting business improvement). 

Taking it all a bit further it’s also unfortunate that most EA efforts top out at business processes (rather than capabilities) and so are a pure expression of how things get done rather than a sensible framework for enabling decisions about what should be done (I enjoyed Steve Jones’s post about this very issue).

Before I get flamed to the extent that I am consumed by a terrible conflagration I have to say a couple of other things; are the aims of EA crazy (i.e. to understand and govern the organisation)?  Umm, no.  Should we still be seeking to do this?  Umm, yes.  Does the kind of top down, all encompassing approach taken by most EA efforts actually work?  Umm, no.  By saying that EA is no longer useful I’m basically saying that the concept is absolutely right, it is more necessary than ever in a world of services – indeed services could be the missing abstraction that finally fill the gap between intent and execution – but that initial efforts in this direction didn’t take sufficient notice (indeed why would they have) of unbundling and federation.  So I believe that EA needs to evolve to help govern the portfolio of capabilities required by an organisation to function and to include some light touch policies that specify how they should work together.  Below this level we start again, essentially with a smaller EA for each capability, recognising federation and unbundling will remove our ability to control every aspect of the way that an organisation works centrally and from top to bottom – EA essentially needs to become explicitly fractal.

Either way I’m not sure that EA is the next big thing, but then again maybe I’ve just dreamt all of this and the future is bright etc.

Killer, Gorilla, Guerilla SOA

The Proposition

Just read an interesting post by Jim Webber about a lighter weight technique for SOA implementation that he calls Guerilla SOA.  This spurred a number of comments from people such as Joe McKendrick, Tony Baer and ZapThink’s Jason Bloomberg.  Generally the comments were positive excepting the usual concerns about bottom up implementations leading nowhere.

Guerilla Campaigns Need Direction

As I’ve discussed previously I’m a big fan of thinking broadly but starting small, since with difficult to grasp concepts like SOA we have to rapidly and continually deliver benefit to generate buy in from our business colleagues.  As a result I am intrigued and broadly supportive of the concept of guerilla SOA implementation.  The key thing in my view, however, is to start in the right way by concentrating initially on getting a broad view of the business capabilities needed by the enterprise through a rapid approximation and then use these as a governing framework for the implementation of services within the organisation. I’m a firm believer in mixing systematic design with a dash of chaos – essentially govern the top level business capabilities required by the organisation but then delegate responsibility for the realisation of these downwards (and since this is a fractal process there are a limited number of capabilities at each level and as such the decision making process is made simpler by restricting the number of abstractions that any one group of people need to manage).  As a result we allow capability owners to realise services any way that they want – potentially meaning that many replicated services spring up across the organisation within the control of different capabilities – since the only thing we care about is that we have the ‘right’ capabilities and that these perform to the cost and quality metrics that we require.  In this context sharing of services at lower levels of granularity will be an emergent property caused by attempts to improve the cost effectiveness and quality of capabilities rather than something that is planned for from day one (which requires a deep and complex understanding of how everything in the organisation works from top to bottom – an impossible task despite the best efforts of many current enterprise architecture approaches).  Equally importantly in this context, attempting to force capability owners to share smaller granularity services by looking for ‘reuse’ often damages these business capabilities in subtle ways that are impossible to guess ahead of time – as a result independence with future emergent sharing opportunities based on metrics is a more appropriate mechanism IMHO (which now that I’ve read that sounds a bit like the agile principle of only building what you need – hadn’t really thought about it within a business context like that before – gives a whole different spin to the hackneyed term ‘agile business’).

Executing Your Campaign

Now the question is how this relates to the discussion of guerilla SOA?  I would argue that the notions of business capabilities and guerilla SOA are orthogonal and complementary.  Essentially you map out the broad ‘shape’ of the organisation (your campaign goals) using business capabilities – consider this the horizontal dimension – but then concentrate on federated projects (guerilla operations) that maximise value in the improvement of these capabilities and fill them out using agile implementation styles, essentially rapidly implementing elements of capabilities with focused projects – consider this the vertical dimension.  The reality is that no business design (or indeed guerilla campaign) can ever be correct and pre-planned from top to bottom at any point in time and so the key is to rapidly approximate the systematic capabilities needed by the organisation and then get on with implementing value in a stepwise fashion.  Your governance process will need to be able to cope with changes – e.g. versioning of capabilities, business messages and everything else – as your transition to SOA proceeds but there is no way to nail all of this up front and for ever and so you’d better get used to that and move on.  Essentially anyone who comes to you and suggests a top-down process re-engineering exercise should be politely shown the door, since such exercises take years, are always out of date and never actually deliver any sustainable value.  Let’s call this top down approach “Killer SOA” as it is guaranteed to go nowhere and will quickly lead to a morass of paperwork that scuppers your attempts at SOA.  Conversely however just pitching in like a gorilla without any context will make you look busy and actually deliver some stuff but it won’t really add up to anything that makes a significant difference to your organisation – the sum of the parts will really be just the parts (and it’ll be more complex and expensive to deliver, operate and manage).  Let’s call this bottom up approach “Gorilla SOA” as we can imagine it as a chase for the next banana without any concern for what such plundering might do to the environment that sustains us.  To me Guerilla SOA takes the best approach possible, relying on a combination of a broad plan and lots of loosely coordinated and federated cells with the power to implement in a situationally appropriate way.  

Enable Guerillas to Improvise in Realising Goals

This approach to business capability realisation is also applicable in the technology space as well.  I am glad to see Jim stating (and an increasing number of other people realising) that technology as architecture (e.g. ESBs/application servers/whatever) is both meaningless and dangerous.  As I’ve discussed in the past trying to force centralised architectural approaches onto an essentially heterogeneous business environment (heterogeneous from a requirements perspective (e.g. QoS or cost requirements) as well as a technical one) leads to an environment that is fit for nobody.  Architectural approaches should concentrate on the space between capabilities (i.e. standards that support interoperability (or an overall loose coordination of cells)) and on the governance of the required outputs (i.e. capability metricisation and performance management (or realisation of strategic operations)) and enable capability owners (and guerillas) to maximise existing assets or new approaches that best map to their goals and aspirations rather than on trying to use technology as an excuse for architecture (I have to also admit to having a particular distaste for ‘ESBs’ which in most incarnations are just hub-and-spoke message hubs with proprietary transport which undermine the very notions of SOA – i.e. federation, standardisation and interoperability – but I guess that’s another post, grrr ).

It’s the Difference Between Killer, Gorilla and Guerilla that Matters

In the broader context the question is whether Jim’s comments reflect an anarchic and directionless dead-end of random service implementation or a dynamic and manageable way to break the SOA deadlock within organisations.  To me the question you have to ask yourself is whether you are a Killer that drowns your organisation in paperwork, a Gorilla who just wades into SOA in an elemental fashion with little thought beyond the moment or whether you are a Guerilla who is part of a larger group of loosely federated fighters striving to realise a shared vision. 

P.S.

I realise that many people refer to ‘Guerilla SOA’ as ‘Middle Out’ (see Nick Malik for an example).  To me the two terms describe the same thing but whereas ‘Guerilla’ makes me think of glamour, adventure and action ‘Middle Out’ makes me think of the male aging process and brings to mind increasing flabbiness around the midriff and balding.  Just sounds less exciting doesn’t it, especially when we’re trying to convince people to follow us into action, LOL