Archive | SOA RSS feed for this section

Enterprise Architecture Top to Bottom

2 Dec

JP Morgenthal published an interesting post on his blog recently relating to the futility of trying to map out every facet of an enterprise architecture.  I wholeheartedly agree with his sentiments and have spoken on this issue in the past – albeit in a slightly different context (and also in discussing evolution and IT, actually).  I also feel strongly that EA practitioners should be focused far more on enabling a deeper understanding of the purpose and capabilities of the enterprises they work in – to facilitate greater clarity of reasoning about strategic options and appropriate action – rather than taking on an often obstructive and disconnected IT strategy and governance role (something that was covered nicely by Neil Ward-Dutton last week).  For all of these reasons I totally agreed with JP’s assertion that we should only pursue absolute detail in those areas that we are currently focused on.  This is certainly the route we took in my previous role in putting together an integration architecture for two financial services companies.

The one area where I think we can add to JP’s thoughtful consideration of the issues is that of developing useful abstractions of the business architecture as pivotal reasoning assets.  In pursuing the work I allude to we developed a business capability map of the enterprise that allowed us to divide it up into a portfolio of ‘business components’.  These capabilities allowed us to reason at a higher level and make an initial loose allocation of the underlying implementation assets and people to each (and given that both I and EA were new to the organisation when we started I even had to ‘crowdsource’ a view of the assets and their allocation to capabilities from across the organisation to kick start the process).  In this sense there was no need at the outset to understand the details of how everything linked together (either across the organisation or within individual capabilities) but rather just the purpose and broad outcomes of each capability.  This is an important consideration as it allowed us to focus clearly on understanding which capabilities needed to be addressed to respond to particular issues and also to reason about and action these changes at a more abstract level (i.e. without becoming distracted by – and lost in – the details of the required implementation).  In this sense we could concentrate not on understanding the detail of every ‘horizontal’ area as a discrete thing – so everything about every process, infrastructure, data or reward systems along with the connections across them all – but rather on building a single critical horizontal asset (i.e. the business capability view) that allowed us to reason about outcomes at an enterprise level whilst only loosely aligning implementation information to these capabilities until such a time as we wanted to make some changes.  At that stage specific programmes could work with the EA team to look much more specifically at actual relationships along with the implementation resources, roles and assets required to deliver the outcomes.  Furthermore the loosely bounded nature of the capabilities meant that we could gradually increase the degree of federation from a design and implementation perspective without losing overall context.

Overall this approach meant that we did not try to maintain a constant and consistent view of the entire enterprise within and across the traditional horizontal views – along with the way in which they all linked together from top to bottom – but only a loose view of the overall portfolio of each with specific contextualisation provided by an organising asset (i.e. the capability model).  In this context we needed to confirm the detailed as-is and to-be state of each capability whenever we wanted to action changes to its outcomes – as we expended little effort to create and maintain detailed central views – but this could be largely undertaken by the staff embedded within the capability with support and loose oversight from the central EA team.  In reality we kept an approximate portfolio view of the assets in the organisation (so for example processes, number of people, roles, applications, infrastructures and data) as horizontal assets along with the fact that there was some kind of relationship but these were only sufficient to allow reasoning about individual capabilities, broad systemic issues or the scale of impact of potential changes and were not particularly detailed (I even insisted on keeping them in spreadsheets and Sharepoint – eek – to limit sophistication rather than get sucked into a heavy EA tool with its voracious appetite for models, links and dependencies).

I guess the point I wanted to make is that my own epiphany a few years ago related to the fact that most people don’t need to know how most things work most of the time (if ever) and that trying to enable them to do so is a waste of time and a source of confusion and inaction.  It is essentially impossible to create and then manage a fixed and central model of how an entire enterprise works works top to bottom, particularly by looking at horizontal implementation facets like processes, people or technology which change rapidly, independently and for different reasons in different capabilities.  In addition the business models of capabilities are going to be very diverse and ‘horizontal’ views often encourage overly simplistic policies and standards for the sake of ‘standardisation’ that negatively impact large areas of the business.  Throw in an increasing move towards the cloud and the consumption of specialised external services and this only becomes more of an issue.  In this context it is far more critical to have a set of business architecture assets at different levels of abstraction that allow reasoning about the purpose, direction and execution strategy of the business, its capabilities and their implementation assets (this latter only for those capabilities you retain yourself in future).  These assets need to be explicitly targeted at different levels of abstraction,  produced in a contextually appropriate way and – importantly – facilitate far greater federation in decision making and implementation to improve outcomes.  Effectively a framework for understanding and actionable insight is far more valuable than a mass of – mostly out of date – data that causes information overload, confusion and inaction.  An old picture from a few years ago that I put together to illustrate some of these ideas is included below (although in reality I’m not sure that I see an “IT department” continuing to exist as a separate entity in the long term but rather a migration of appropriate staff into the enterprise and capability spaces with platforms and non-core business capabilities moving to the cloud).

guerilla

In terms of relinquishing central control in this way it is possible that for transitional business architectures – where capabilities remain largely within the control of a single enterprise as today – greater federation coupled with a refined form of internal crowd sourcing could enable each independent model to be internally consistent and for each independent model to be consistent with the broader picture of enterprise value creation.  I decided to do something else before getting to the point of testing this as a long term proposition, however, lol (although perhaps my former (business) partner in crime @pcgoodie who’s just started blogging will talk more about this given that he has more staying power than me and continues the work we started together, lol).  Stepping back, however, part of the value in moving to this way of thinking is letting go and viewing things from a systems perspective and so the value of having access to all the detail from the centre will diminish over time.

In the broader sense, though, whilst I first had a low grade ‘business services as organisation’ epiphany whilst working at a financial services company in 2001 most of this thinking and these ways of working were inspired not by being inside an enterprise but rather subsequently spending time outside of one.  Spending time researching and reflecting on the architectures, patterns, technologies and – more importantly – business models related to the cloud made me think more seriously about the place of an enterprise in its wider ecosystem of value creation and the need to concentrate completely on those aspects of the ecosystem that really deliver its value.  In the longer term whilst there are many pressures forcing an internal realignment to become more customer-centric, valuable or cost effective, the real pressure is going to start building from the outside; once you realise that the enterprise works within a broader system you also start to see how the enterprise itself is a system, with most of its components being pretty poor or misaligned to the needs of the wider ecosystem and its consumers.  At this point you begin to realise that you have to separate the different capabilities in your organisation and use greater design thinking, abstraction and federation, giving up control of the detail outside of very specific (and different) contexts depending on your purview.  At that stage you can really question your need to be executing many capabilities yourself at all, since the real promise of the cloud is not merely to provide computing power externally but rather to enable businesses to realise their specialised capabilities in a way that is open, collaborative and net native and to connect these specialisations across boundaries to form new kinds of loose but powerful value webs.  Such an end game will be totally impossible for organisations who continue to run centralised, detail-oriented EA programmes and thus do not learn to let go, federate and use abstraction to reason, plan and execute at different levels simultaneously.

What’s the Future of SOA?

9 Nov

EbizQ asked last week for views on the improvements people believe are required to make SOA a greater success.  I think that if we step back we can see some hope – in fact increasing necessity – for SOA and the cloud is going to be the major factor in this.

If we think about the history of SOA to date it was easy to talk about the need for better integration across the organisation, clearer views of what was going on or the abstract notion of agility. Making it concrete and urgent was more of an issue, however. Whilst we can discuss the ‘failure’ of SOA by pointing to a lack of any application of service principles at a business level (i.e. organisationally through some kind of EA) this is really only a symptom and not the underlying cause. In reality the cause of SOA failure to date has been business inertia – organisations were already set up to do what they did, they did it well enough in a push economy and the (understandable) incentives for wholesale consideration of the way the business worked were few.

The cloud changes all of this, however. The increasing availability of cloud computing platforms and services acts as a key accelerator to specialisation and pull business models since it allows new entrants to join the market quickly, cheaply and scalably and to be more specialised than ever before. As a result many organisational capabilities that were economically unviable as market offerings are now becoming increasingly viable because of the global nature of cloud services. All of these new service providers need to make their capabilities easy to consume, however, and as a result are making good use of what people are now calling ‘apis’ in a web 2.0 context but which are really just services; this is important as one of the direct consequences of specialisation is the need to be hooked into the maximum number of appropriate value web participants as easily as possible.

On the demand side, as more and more external options become available in the marketplace that offer the potential to replace those capabilities that enterprises have traditionally executed in house, so leaders will start to rethink the purpose of their organisations and leverage the capabilites of external service providers in place of their own.

As a result cloud and SOA are indivisable if we are to realise the potential of either; cloud enables a much broader and more specialised set of business service providers to enter a global market with cost and capability profiles far better than those which an enterprise can deliver internally. Equally importantly, however, they will be implicitly (but concretely) creating a ‘business SOA catalogue’ within the marketplace, removing the need for organisations to undertake a difficult internal slog to re-implement or re-configure outdated capabilities for reuse in service models. Organisations need to use this insight now to trigger the use of business architecture techniques to understand their future selves as service-based organisations – both by using external services as archtypes to help them understand the ways in which they need to change and offer their own specialised services but also to work with potential partners to co-develop and then disaggregate those services in which they don’t wish to specialise in future.

Having said all that to set the scene for my answer(!) I believe that SOA research needs to be focused on raising the concepts of IT mediated service provision to a business level – including concrete modelling of business capabilities and value webs – along with complex service levels, contracts, pricing and composition – new cloud development platforms, tooling and management approaches linked more explicitly to business outcomes – and which give specialised support to different kinds of work – and the emergence of new 3rd parties who will mediate, monitor and monetise such relationships on behalf of participants in order to provide the required trust.

All in all I guess there’s still plenty to do.

Differentiation vs Integration (Addenda)

22 Jun

After completing my post on different kinds of differentiation the other day I still had a number of points left over that didn’t really fit neatly into the flow of the arguments I presented.  I still think some of them are interesting, though, and so thought I’d add them as addendums to my previous post!

Addendum 1

The first point was a general feeling that ‘standardisation’ is a good thing from an IT perspective.  This stemmed from one of Richard’s explicit statements that:

“Many people in the IT world take for granted that standardization (reduction in variety) is a good thing”

Interestingly it is true to say that from an IT perspective standardisation is generally a good thing (since IT is an infrastructural capability).  Such standardisation, however, must allow for key variances that allow people to configure and consume the standardised applications and systems in a way that enables them to reach their goals (so they must support configuration for each ‘tenant’).  Echoing my other post on evolution – in order to consider this at both an organisational and a market level – we can see that a shift to cloud computing (and ultimately consumption of specialised business capabilities across organisational boundaries) opens up a wider vista than is traditionally available within a single company.

In the traditional way of thinking about IT people within a single organisation are looking to increase standardisation as a valid way of reducing costs and increasing reliability within the bounds of a single organisation’s IT estate.  The issue with this is that such IT standardisation often forces inappropriate standardisation – both in terms of technology support and change processes – on capabilities within the business (something I talked about a while ago).  Essentially the need to standardise for operational IT efficiency tries to override the often genuine cost and capability differences required by each business area.  In addition on-premise solutions have rarely been created with simple mass-configuration in mind, requiring expensive IT customisation and integration to create a single ‘standard’ solution that cannot be varied by tenant (tenant in this case being a business capability with different needs).  Such tensions result in a constant war between IT and the single ‘standard’ solution they can afford to support and individual business capabilities and the different cost and capability requirements they have (which often results in departmental or ‘shadow’ IT implemented by end users outside the control of the IT department).

The interesting point about this, however, is that cloud computing allows organisations to make use of many platforms and applications without a) the upfront expenditure usually required for hardware, training and operational setup and b) the ongoing operational management costs.  In this instance the valid reasons that IT departments try to drive towards standardisation – i.e. reducing the number of heterogeneous technologies they must deploy, manage and upgrade – largely disappear.  If we also accept that IT is essentially infrastructural in nature – and hence provides no differentiation – then we can easily rely on external technology platforms to provide standardisation and economies of scale on our behalf without having to mandate a single platform or application to gain these efficiencies.  At this point we can turn the traditional model on its head – we can choose different platforms and applications for each capability dependent on its needs without sacrificing any of the benefits of standardisation (subject to the applications and platforms supporting interoperability standards to facilitate integration).  Significant and transformational improvements enabled by capability-specific optimisation of the business is therefore (almost tragically) dependent on freeing ourselves from the drag of internal IT.

Addendum 2

Richard also highlighted the fact that there is still a strong belief in many quarters that ‘business architecture’ should be an IT discipline (largely I guess from people who can’t read?).  I believe that ‘business’ architecture is fundamentally about propositions, structure and culture before anything else and that IT is simply one element of a lower level set of implementation decisions.  Whilst IT people may have a leg up on the required ‘structured thinking’ aspects necessary to think about a businesses architecture I feel that any suggestion that business owners are too stupid to design their own organisations – especially using abstraction methods like capabilities – seems outrageous to me.  IT people have an increasingly strong role to play in ‘fusing’ with business colleagues to more rapidly implement differentiating capabilities but they don’t own the business.  Additionally, continued IT ownership of business architecture and EA causes two additional issues: 1) IT architecture techniques are still a long way in advance of business architecture techniques and this means it is faster, easier and more natural for IT people to concentrate on this; and 2) The lack of business people working in the field – since they don’t know IT – limits the rate at which the harder questions about propositions and organisational fitness are being asked and tackled.  As a result – at least from my perspective – ‘business architecture’ owned by IT delivers a potential double whammy against progress; on the one hand it leads to a profusion of IT-centric EA efforts targeted at low interest areas like IT efficiency or cost reduction whilst on the other it allows people to avoid studying, codifying and tackling the real business architecture issues that could be major strategic levers.

Addendum 3

As a final quick aside the model that I discussed for viewing an organisation as a set of business capabilities gives rise to the need for different ‘kinds’ of business architects with many levels of responsibility.  Essentially you can be a business architect helping the overall enterprise to understand what capabilities are needed to realise value streams (so having an enterprise and market view of ‘what’ is required) through to a business architect responsible for how a given capability is actually implemented in terms of process, people and technology (so having an implementation view of ‘how’ to realise a specific ‘what’).  In this latter case – for capabilities that are infrastructural in nature and thus require high standardisation – it may still be appropriate to use detailed, scientific management approaches.

Evolution and IT

3 Jun

This is a subject that has been on my mind a lot lately as I recently read an astounding book by Eric D. Beinhocker called “The Origins of Wealth”.  It was astounding to me for the way in which Beinhocker imperiously swept across traditional economic theories based on equilibrium systems, critiqued the inherent weaknesses of such theories when faced with real world scenarios and then hypothesised the use of the evolutionary algorithm as a basis for a fundamental shift to what he called ‘complexity economics’.  I’m going to return to discuss some of the points from this book – and the way in which they resonated with my own thoughts around business design, economic patterns and technology change – but for today I just wanted to comment on a post by Steve Jones where he raises the issue of evolution in the context of IT systems.

Steve’s question was whether we should “reject evolution and instead take up arms with the Intelligent design mob”.  His thoughts have been influenced by the writing of Richard Dawkins, in particular the oft-times contrast between the apparent elegance of the external appearance of an animal (including its fitness for its environment) with the messy internals that give it life.  Steve suggests that he sees parallels in the IT world and brings this around to issues with the way in which a shift to service-based models often creates unfounded expectations on internal agility:

“The point is that actually we shouldn’t sell SOA from the perspective of evolution of the INSIDE at all we should sell it as an intelligent design approach based on the outside of the service. Its interfaces and its contracts. By claiming internal elements as benefits we are actually undermining the whole benefits that SOA can actually deliver.”

In the rest of the post and into the comments Steve then extends this argument to call for intelligent design (of externals) in place of evolution:

“The point I’m making is that Evolution is a bad way to design a system the whole point of evolution is that of selection, selection of an individual against others. In IT we have just one individual (our IT estate) and so selection doesn’t apply.”

My own feeling is that there isn’t a direct 1:1 relationship in thinking about evolution and the difficulties of changing the internals of a service in the way that Steve suggests.  I believe that evolution is a fractal algorithm whose principles apply equally to the design of business capabilities, service contracts and code.  To think about this more specifically I’d like to consider a number of his points after first considering evolution and how we frame it more broadly from a market and enterprise context.

What is evolution?

Evolution is an algorithm that allows us to explore large design spaces without knowing everything in advance.  It allows us to try out random designs, apply some selection criteria and then amplify those characteristics of a design that are judged as ‘fit’ by the environment (i.e. the selection criteria).  In the natural world evolution throws up organisms that have many component traits and success is judged – often brutally – by how well these traits enable an animal to survive in the environment in which it exists.  Within an individual species there will be a particular subset of traits that define that species (so traits that govern size, speed or vision for instance).  Individuals within a species who have the most desirable instances of these traits will be better equipped to survive, the mating of these individuals will merge their desirable traits and over time the preponderance of the most effective traits will therefore increase in the population overall.  As a result evolution creates a number of designs and uses a selection algorithm to more rapidly arrive at designs that are ‘good enough’ to thrive within the context of the environment in which they exist.  It is a much more rapid method of exploring large design spaces than trying to think about every possible design, work out the best combination of traits and then create the ‘perfect’ design from scratch (i.e. “intelligently” design something without a full understanding of the complexities of the selection criteria and hence what will be successful).

Enterprises and evolution

In Beinhockers book he uses a ‘business’ as the unit of selection that operates within the evolutionary context of the market.  Those businesses with successful traits are chosen by consumers and thus excel.  These traits – whether they be talent strategies, process strategies or technology strategies – are then copied by other businesses, replicating and amplifying successful traits within the economic system. 

I believe that this is the best approximation that we can use in the – rather unsystematic – businesses that exist today but that we can use systematic business architecture to do better.  I have often written about my belief in the need for companies to become more adaptable by identifying and then reforming around the discrete business capabilities they require to realise value.  Such capabilities would form a portfolio of discrete components with defined outcomes which could then be combined and recombined as necessary to realise systematic value streams. 

Such a shift to business capabilities will allow an enterprise to adapt its organisation through the optimisation and recombination of components; whilst at this stage of maturity Beinhocker’s hypothesis of the ‘business’ as the element of selection remains sound (since capabilities are still internal and not individually selectable as desirable traits) we can at least begin to talk about capabilities and the way in which we combine them as the primary traits that need to be ‘amplified’ to increase the fitness of the design of our business. 

Inside Out 

Whilst realigning internal capabilities is a worthwhile exercise in its own right, evolutionary systems also tend to exhibit long periods of relative stability punctuated by rapid change as something happens to alter the selection criteria for ‘fitness’.  The Web and related techniques for decomposition – such as service-orientation – have made it possible to consume external services as easily as internal services.  Business capabilities can thus be made available by specialised providers from anywhere in the world in such a way that they can be easily integrated with internal capabilities to form collaborative value webs.  We can therefore view the current convergence of business architecture, technology and a mass shift to service models as a point of ‘punctuated equilibrium’. 

In this environment continuing to execute capabilities that are non-differentiating will cease to be an attractive option as working with specialised providers will deliver both better outcomes and more opportunities for innovation.  From an evolutionary perspective our algorithm will continue to select those organisations that are most fit (as judged by the market) and those organisations will be those with the strongest combination of traits (i.e. capabilities).  Specialised, external capabilities can be considered to be more attractive ‘traits’ due to their sharp focus, shorter feedback loops and market outlook; they will thus be amplified as more organisations close down their own internal capabilities and integrate them instead, a kind of organisational mutation caused by the combination of the best capabilities available to increase overall fitness.    Enterprises working with limited, expensive and non-differentiating internal capabilities will risk extinction.

Once this shift reaches a tipping point we discover that business capabilities become the unit of market selection since they are now visible as businesses in their own right.  Whilst this could be considered pedantry – as a ‘business’ is still the unit of selection even though what we consider a business has become smaller – there is an important shift that happens at this point.  Essentially as business capabilities become units of selection in their own right the ‘traits’ for selection and amplification of their services become a combination of their own internal people, process and technology capabilities plus the quality of the external capabilities they integrate.  Equally importantly they have to act as businesses – rather than internal, supporting organisations – and support the needs of many customers – and hence support mass customisation.  This will mean that they will have many more consumers than internal support functions would ever have had and the needs of these consumers could be both very different and impossible to guess in advance; there will be new opportunities to rapidly improve their services based on insight from different industries, orthogonal areas and new collaborations.  An ability to respond to these new opportunities by changing their own capabilities or finding new partners to work with will be a significant factor in whether these capabilities thrive and are thus judged as ‘fit’ by the selection criteria of the market.  An ability to evolve externally to provide the ‘right’ services will thus be a core competency required in the new world. 

What has this got to do with services?

The basic points I’m making here are that evolution acts at the scale of markets and is a process that we participate in rather than a way of designing.  We design our offers using the best knowledge that we have available but the market will decide whether the ‘traits’ we exhibit fit the selection criteria of the environment.  Business capabilities can become the ‘traits’ that make particular market offers (or businesses) fit for selection or not by having a huge influence over the overall cost, quality and desirability of a particular offer.  From a technology perspective such capabilities will in reality need to offer their services as services in order for them to be easily integrated into the overall value webs of their customers and partners; in many cases there may be a 1:1 mapping between the business capability and the service interface used to consume it.  In that sense services are just as much a driver of fitness in the overall ecosystem and their interface and purpose will inevitably need to change as the overall ecosystem evolves.  Hence it is not simply a question of ‘fixing’ interfaces and ‘evolving’ internals; the reality is that the whole market is an evolutionary system and businesses – plus the services they offer for consumption –will need to continually evolve in order to remain fit against changing selection criteria.

Intelligent design or evolution

The core question raised by Steve is whether ‘evolution’ has any place in our notion of service design.  In particular:

“The point I’m making is that Evolution is a bad way to design a system the whole point of evolution is that of selection, selection of an individual against others. In IT we have just one individual (our IT estate) and so selection doesn’t apply.”

Is evolution a ‘bad’ designer?

I do not believe that evolution is either a good or a bad designer but it is a very successful one.  Evolution is an algorithm that takes external selection criteria, applies them and amplifies those traits that are most successful in meeting the criteria.  It is brilliant at evaluating near-infinite design spaces (such as living organisms or markets) and continually refining designs to make them fit for the environmental selection criteria in play.

If I read Steve’s post correctly, however, he actually isn’t objecting to the notion of evolution per se – since at the macro level it is a market process in which we are all involved and not a conscious way of designing our services – but rather to a lack of design being labelled an ‘evolutionary’ approach. 

What is ‘evolutionary’ design?

In the majority of cases when people talk about ‘evolution’ in the context of systems they really mean that they want to implement as quickly and cheaply as possible and then ‘evolve’ the system.  Often vendors encourage this behaviour by promising that new technologies are so rapid as to make changes easy and inexpensive.  Such approaches often eschew any attempt at formal design, choosing instead to implement in isolation and then retro-integrate with anything else on a case by case basis.  I have often seen Steve talk about the evils of generating WSDL from code and I imagine that this is the sort of behaviour that he is classifying as ‘evolutionary’ changes to internals.

Is this a good or a bad approach?  From an evolutionary perspective we can say that we do not care.  Given that we are talking about evolution in its true sense the algorithm would merely continue to churn through its evaluation of services, amplifying successful traits.  It is just that such behaviour might have some unrecognised issues: firstly evolution would have to work for longer to bring a service to a point at which it is ‘fit’, secondly the combination of all of these unfit services means that there is a multiplier effect to evolving the ecosystem of services to a point at which it is fit overall and thirdly whilst all of this goes on at a micro level the fitness of the enterprise against the selection criteria of the market might be poor due to the unfitness of some of its major ‘traits’.

Intelligence in design

Whilst a lack of design might extend the evolutionary process to a point at which it is unlikely that a business could ever become fit before it became extinct, an assumption that we can design service interfaces that are fixed also ignores the reality of operating in a complex evolutionary system (like a business). 

Creating a ‘perfect’ service from scratch is a very difficult thing to do as even within the bounds of a single organisation we cannot know all of the potential uses that might come to pass.  We can however use the best available data to create an approximation of the business capabilities and resulting services required in order to try and speed up the evolutionary process by reducing the design space it has to search.  Hence the notion that we use an evolutionary process of service design (a bit like I discussed here) is an important one; often people will not know what good looks like until they see something.  Whilst I therefore accept that we can start with an approximation of the capabilities (and services) we believe we will need we have to accept that these will evolve as we gain experience and exposure to new use cases.  

From this perspective I don’t agree with the literal statement that Steve has made; it is not about intelligent design vs evolution but rather about intelligence of design to support the evolutionary process.  As I stated previously markets are fundamentally evolutionary systems and therefore our businesses – and the business capabilities and services that represent their traits – are assessed by the evolutionary algorithm for fitness against market selection criteria.  We are not dumb observers in this process, however, and must fight to create offers that are attractive to the market along with supporting organisations that enable us to do it at the right price and service levels.  We can apply our intelligence to this process to increase our chances of success but a key element will be to understand that our enterprises will increasingly become a value web of individual capabilities, that it is the combination of our capabilities that is judged and that we must therefore design our organisations to evolve by adopting successful traits to improve our overall fitness.  As a result we should not expect the evolutionary process to do our work for us – by choosing not to apply any intelligence in design – but we should also not assume that evolution has no place in design given that meeting its demands is becoming the primary requirement of business architecture.

Macro evolution in the economy

Stepping back and taking an external perspective leads us to realise that it is also untrue to say that we only have one individual (in terms of a single IT estate) and that there is nothing to select against; in reality even today we are competing for selection against businesses with other IT estates and thus our ‘traits’ (in the form of our IT) are already a major factor in deciding our fitness (and thus our ability to be ‘selected’ by the evolutionary algorithm of the market).  If we factor in the emerging discontinuities we see as part of the ‘punctuated equilibrium’ process it only makes things worse; the specific IT we have within specific business capabilities will have a large impact on the fitness of these capabilities to survive.  In that context continually evolving our business capabilities (and with them the IT and software services that enable them) is the only way to ensure future success.

More importantly as we look at the wider picture of the position of our business capabilities within the market as a whole so our unknowns become more acute and the more that we can only rely on selection and amplification (i.e. evolution) to guide us in shaping them.  Looking beyond the boundaries of our single organisation we have to consider the fact that all of our services will exist in a market ecosystem many of whose needs and usage we are even less equipped to know in advance.  There will often be new and novel ways in which we can change our services to meet the emerging needs of customers and partners and in this way the overall ecosystem itself will evolve.  As a result selection is the only way in which design can occur in an ecosystem as complex as a market where there are many participants whose needs are diverse.  Nobody can ‘intelligently design’ a whole market from top to bottom.  Furthermore the market – as an evolutionary system – will be subject to a process of ‘punctuated equilibrium’ meaning that sudden changes in the criteria used to judge fitness can occur.  From an IT perspective the shift towards service models such as cloud computing could be considered to be one such change, since it changes the economics of IT from one based on differentiation based on ownership to one based on universal access.  Such changes could be considered ‘revolutionary’ as the carefully crafted and scaled business models created during a period of relative stability cease to be appropriate and new capabilities have to be developed or integrated to be successful.  This is one area where I disagreed with Steve’s comment about the relationship between revolution and evolution:

“The point of revolutionary change is that it may require a drop back to the beginning and starting again. This isn’t possible in an evolutionary model.”

Essentially revolutionary change often happens in evolutionary systems – evolution is always exploring the design space and changes in the environment can lead to previously uninteresting traits becoming key selection criteria.  In this case ‘revolutionary change’ is a side-effect of the way in which evolution starts to amplify different traits due to the changes in selection criteria.  In the natural world such changes can lead to catastrophic outcomes for whole species whose specialisations are too far removed from the new selection criteria and this can also happen to businesses (it will be interesting to see how many IT companies survive the shift to new models, for instance).  Evolution also allows the development of new ‘traits’ that make us sustainable, however, and therefore can support us in surviving ‘revolutionary’ changes if we have sufficient desirable ‘traits’ to prevent total collapse.  The trick is to understand how you can evolve at the macro level to incorporate the changes that have occurred in the selection criteria of your market and to realign your capabilities as appropriate.  Often the safest way to do this is to have different services on offer that try different combinations of traits, hence keeping sensors within the environment to warn you of impending changes. 

Summary

As a result there is no question that both evolution and intelligence in design have a place in the creation of sustainable architectures (whether macro business architectures or micro service architectures).  We have to be precise in the ways in which we use this language, however; it is not sufficient to label a lack of design as ‘evolution’ (which I believe was Steve’s core point).  Evolution is a larger, exogenous force that shapes systems by highlighting and amplifying desirable traits and not something that we can rely on to reliably fix our design issues without an infinite amount of time and change capability.  We therefore need to apply intelligence to the process of design – even when there is great uncertainty – to try and narrow down the design space to minimise the amount we have to rely on evolution to arrive at a viable ‘system’; even once we get to this point, however, we need to be aware of the fact that evolution is an ongoing process of selection and amplification and design our business architectures with the flexibility necessary to recognise this fact.

More broadly I believe that we can also look at the application of ‘intelligence’ and ‘evolution’ as a matter of scale;  we can design individual services with a fair degree of intelligence, we can design our business capabilities with some fair approximations and then rely on evolution to improve them but we can only rely on evolution to shape the market itself and thus the selection criteria that define our participation.  For this reason strategies that stress adaptability (i.e. an ability to evolve in response to changing selection criteria) have to take precedence over strategies that stress certainty and efficiency.

Differentiation vs Integration

12 May

Read an interesting post today by Richard Veryard looking at the current state of business architecture.  In particular Richard was musing on the traditional dichotomy between standardisation and integration vs differentiation and autonomy from a business perspective.  Essentially Richard was using the Clive Finkelstein  adapted version of the two-by-two matrix from Ross, Weill and Richardson’s “Enterprise Architecture as Strategy”, below.

 

 

Firstly Richard points out that the general tendency is for everyone to think that ‘unification’ (so highly standardised, highly integrated capabilities) is the best place to be.  I’d first like to say that this is rubbish.  The reality is that an organisation – whether they know it or not – consists of many different business capabilities and the best position depends entirely on the capability in question. 

To start to break this down against the two axes, lets broadly examine integration and then standardisation:

  • If we consider a typical enterprise to increasingly be a collection of business capabilities that reside inside and outside the boundaries of a single firm then we can see that integration is indeed becoming far more important.  Integration in this context, however, does not require the traditional tight integration of an end to end business process but rather the loose integration of autonomous service providers that collaborate to deliver value.  In this sense whilst ‘value stream’ integration may be high, ‘business process’ integration needs to be kept deliberately low;
  • On the other hand differentiation of business capabilities is also claimed to be becoming more important as a result of increasing commoditisation and everyone is trying to understand how to be more agile in their offerings, more responsive to their customers and more relevant to their local markets.  Agility often leads to lower levels of integration and standardisation, however, as smaller groups seek the autonomy necessary to optimise “their” part of the business as it operates in the context of their location and stakeholders.

As a result the simple answer would seem from a business side demand perspective that ‘diversification’ (as requiring low business process integration and being potentially the greatest enabler of agility) is the new top right whereas most IT people are still aiming for ‘unification’. How do we resolve this seeming paradox?

Business Process (dis)Integration

The first issue to tackle is the seeming requirement of ‘business process integration’ as an indicator of increasing maturity in the graph.  Such language was potentially appropriate when business processes were seen to be the highest level of abstraction available to a business architect and when internal departments were the only major source of supply.  In this context it was common to try and eliminate waste by “hardcoding” all of the end-to-end process steps required to realise some value for the customer.  In this practice you  would usually have one or more architects who worked out the whole detail of how a process worked end-to-end in terms of who did what, how they did it and when they did it by – Taylorism was rife.  Such practices – whilst still widespread – are now essentially bankrupt at the macro level as a result of new methods and the rise of the web.

Business Capabilities and Extended Value Streams

I’ve talked many times about the use of business capabilities to document the stable structure of an organisation and provide a framework for concentrating on ‘what’ the organisation does rather than getting lost in the mess of ‘how’ it actually does it.  Business capabilities allow us to concentrate on the outcomes we require and map value streams both within and across organisations to coordinate these outcomes but ignore how the value is delivered.  This allows us to federate out the issues of implementation to the owners of each capability and give them the maximum scope for optimising their capability.  This is particularly important when you realise that codification of outcomes gives you options for supply – given that transaction costs have now made use of external capabilities highly desirable – but also requires you to let go of tight control of implementation and trust your chosen partner.  I wrote a long post on this whole subject here for those who are interested.

As a result whilst it might seem pedantic to separate ‘business process’ integration from ‘value stream’ integration I strongly believe that this extra level is needed to differentiate between tight integration of how you do things versus coordination (and hence loose integration) of autonomous outcomes.  This point is highly important in the context of our discussion as it allows us to understand that it is a) possible to identify and decouple the different business capabilities that we require and b) optimise these capabilities in different ways based on their characteristics.   This latter point is crucial as we look at the issue of whether the above matrix is sufficient to categorise emerging models.

No Capability is Created Equal

The first thing we have to realise is that the different business capabilities that implicitly make up an organisation today will respond to different kinds of strategies and thus require different cultural and economic thinking.  Once we have taken the first step and used business architecture to identify these discrete business capabilities we can begin to use this insight to better understand the optimum business architecture of each.

Different ‘business’ types within an Enterprise

Broadly there are four different kinds of businesses emerging as a result of the rise of the web and plummeting transaction costs.

  • Infrastructure businesses: These businesses are essentially business capabilities that respond to economies of scale.  In this sense they need to be highly standardised and have tight integration within their internals – to drive low cost and reliability – but won’t be very innovative or have time for the deep relationships required to meet the needs of individual customers.  Examples of these businesses could be literal platforms – such as a telephony network – or ‘standardised’ services such as HR.  From an economic perspective such business capabilities will be driven to providing services at the lowest possible price (and thus the highest possible scale) but they will also provide relatively stable returns given the broad consumer groups that would leverage them and the difficulties that new entrants would have due to the capital requirements.  It is worth saying that whilst this may look like commoditisation – and thus something to be feared – in this business type the embracing of commoditisation whilst maintaining an ability to operate profitably is actually your differentiation.  From a cultural perspective the business model would require an organisation that is highly standardised, focused and repetitive;
  • Relationship businesses:  These businesses are essentially business capabilities that leverage deep relationships.  Basically relationship-based capabilities rely on deep knowledge of their customers and the market they operate in to bring relevant products and services to their attention at the right time.  These capabilities are less able to be standardised – as they rely on personal customer context – and are also not likely to be strongly innovative as they will generally be responding to the stated needs of their customers – who only know what they know.  Examples of such businesses could be general practitioners or financial advisors.  From an economic perspective these businesses can take advantage of economies of scope – as they focus more on their target customers than on specific product or service niches – and can be very profitable due to the value they are perceived to add.  Culturally the business model would need to be highly flexible to build strong networks and to deliver combinations of products and services that were specifically suited to their target groups;
  • Innovation and commercialisation businesses: These businesses are essentially small, innovation focused capabilities who specialise in IP generation and product and service commercialisation. They will often be one step removed from the demands of current customer need in order to have the time away from delivery pressures and thereby work on breakthrough products and services. Examples of these kinds of companies might be software startups or pharmaceutical research companies.  Their economics is based on intangibles and a long term growth in capital worth from the leverage of their successfully commercialised IP.  Their culture will be very loose, collegiate and focused on research and exploration and many of their endeavours will not result in any tangible success – as a result whilst the rewards for the small number of successful projects are high there is also the fact that most exploration will result in no valuable IP; and
  • Portfolio businesses:  These businesses represent the residual strategy and investment capabilities of large companies and will be run as a separate business type. They will continue to own and manage brands but will replace direct control of delivery capabilities and assets with investment in a portfolio of specialised organisations. Examples of this kind of business might be Virgin or a large IT company such as Fujitsu or IBM.  Their economics will be primarily risk-based, investing money in a range of different organisations across categories in order to leverage their brand, balance their portfolio and maximise their return on capital. As a result they will invest in relationship businesses, infrastructure businesses and innovation businesses in order to get the right mix of stable, low growth returns and high risk, high growth returns.
Differentiation is…. different

Each of these categories of capability needs to maximise its assets in different ways, leading to completely different economics and cultures. I believe that these pressures – coupled with the decreasing transaction costs of working with others – will drive the fragmentation of organisations along these fault lines, since trying to manage them within a single organisational context will necessarily emphasise one whilst sub-optimising the others. Even without this, however it is plain that the different capabilities required to realise an end-to-end value stream need to be managed and optimised in different ways – with management styles, measures and dashboards altered accordingly for each – if we are not to sub-optimise the whole.  As an example, given that infrastructural capabilities are the largest (and often easiest to manage due to their concrete nature), many organisations tend towards management styles and metrics that reflect a concentration on these capabilities; concentrating on efficiency (in order to optimise your infrastructural capabilities), however, will destroy your ability to build deep relationships or to have the detachment necessary to innovate or invest successfully. As a result those organisations which manage to separate these different concerns and specialise – and in so doing create the right culture and economic metrics to underpin appropriate behaviours – will see much greater success than those who continue to operate their organisation with muddled thinking.

There is no Paradox, only Greater Subtlety… and Opportunities

The major implication of these insights is that there is no single ‘strategy’ box that an individual enterprise can be allocated to.  As Richard states in his post (my emphasis):

“….a two-by-two matrix would be misleading. The point isn’t to choose whether you have differentiation and integration or not; the point is to determine how much and what kinds of differentiation  and integration you need.”

I would also add to this understanding where you need that differentiation or integration (i.e. within which capabilities).

If each business is viewed as the portfolio of internal and external capabilities required to realise a given set of value streams then we can envisage a scenario in which each business strategy will be:

  1. An overall statement of intent as captured by the target structure of the enterprise (i.e. its intended portfolio of capabilities, the outcomes they will need to deliver and the sourcing strategy for each); plus
  2. The many federated strategies required by the individual capabilities to meet their outcome obligations. 

In this context tools like the matrix above may be helpful for enterprise strategists and capability owners to select appropriate strategies but only after they have evaluated the ‘type’ of capabilities they have and made prior strategic decisions about core competence (relationships, infrastructure, innovation or portfolio) and divested the implementation of the rest to specialised partners.  Those that remain can then be optimised as suggested based on their nature.  As immediate – but not very well thought through, lol – suggestions: infrastructural capabilities will tend towards ‘replication’ and ideally ‘unification’ strategies, relationship capabilities will tend towards ‘coordination’ strategies, whereas innovation and portfolio capabilities would likely tend towards ‘diversification’ strategies. 

As a result there is no paradox at all, only greater subtlety uncovered by increasing sophistication in the available body of thinking about business models and architecture.  The irony is that not is it only possible for a business to pursue both differentiation and standardisation strategies at the same time but rather that they increasingly must do so if they are to optimise their value streams and be competitive.  To do this, however, they must understand themselves as a portfolio of capabilities that need to be selected, retained or outsourced and then optimised based on their cultural and economic imperatives; unfortunately, however, not many organisations have even begun the process of working this out yet and thus business architects and business architecture are still more like honorary titles for valuable but difficult-to-place individuals than a widespread discipline.

Cloud Platforms and Future Middleware

6 May

I’m going to try and break the habit of a lifetime in this ‘second life’ of my blog and post the odd ‘peppy’ comment on things I’ve seen as well as getting sucked into long analyses :-p

In that spirit I thought I’d just comment on a post I saw today by John Rymer at Forrester; essentially John was expressing some mild disappointment at a discussion about future app servers he was involved in and suggesting that the future of these products needs to be radically different in a connected, cloud environment.  I completely agreed with his points about more lightweight, specialised and virtualised ‘containers’ and this reflected the work I discussed in one of my older posts, where I talked about the need to use virtual templates, lightweight product and framework configurations, specific patterns and metadata plus domain specific languages and factories in pursuit of IT industrialisation.  Such lightweight and specialised containers for service realisation help to make developers more productive but also enable much greater agility and efficiency in resource usage by allowing each such service to change and scale according to its purpose and needs independent of the others.  In this sense I understand the feeling of one person who left a comment who described such platforms in terms of a fabric; this is probably an apt description given that you will have independent, specialised services bound to specific lightweight containers, ‘floating’ on a virtual infrastructure and collaborating with others to realise wider intent.  At heart a lot of John’s post was about simplifying, downsizing and specialising containers for different kinds of services and so I heartily agreed with his sentiments on the matter.

Business Enablement as a Key Cloud Element

30 Apr

After finally posting my last update about ‘Industrialised Service Delivery’ yesterday I have been happily catching up with the intervening output of some of my favourite bloggers.

One post that caught my eye was a reference from Phil Wainwright – whilst he was talking about the VMForce announcement – to a post he had written earlier in the year about Microsoft’s partnership with Intuit.  Essentially one of his central statements was related directly to the series of posts I completed yesterday (so part 1, part 2 and part 3):

“the breadth of infrastructure <required for SaaS> extends beyond the development functionality to embrace the entirely new element of service delivery capabilities. This is a platform’s support for all the components that go with the as-a-service business model, including provisioning, pay-as-you-go pricing and billing, service level monitoring and so on. Conventional software platforms have no conception of these types of capability but they’re absolutely fundamental to delivering cloud services and SaaS applications”.

This is one of the key points that I think is still – inexplicably – lost on many people (particularly people who believe that cloud computing is primarily about providing infrastructure as a service).  In reality the whole world is moving to service models because they are simpler to consume, deliver clearer value for more transparent costs and can be shared across organisations to generate economies of scale.  In fact ‘as a service’ models are increasingly not going to be an IT phenomenon but also going to extend to the way in which businesses deal with each other across organisational boundaries.  For the sale and consumption of such services to work, however, we need to be able to ‘deliver’ them; in this context we need to be able to market them, make them easy to subscribe to, manage billing and service levels transparently for both the supplier and consumer and enable rapid change and development over time to meet the evolving needs of service consumers.  As a result anyone who wants to deliver business capabilities in the future – whether these are applications or business process utilities – will need to be able to ensure that their offering exhibits all of these characteristics. 

Interestingly these ‘business enablement’ functions are pretty generic across all kinds of software and services since they essentially cover account management, subscription, business model definition, rating and billing, security, marketplaces etc etc (i.e. all of the capabilities that I defined as being required in a ‘Service Delivery Platform’).  In this context the use of the term ‘Service Delivery Platform’ in place of cloud or PaaS was deliberate; what next generation infrastructures need to do is enable people to deliver business services as quickly and as robustly as possible, with the platforms themselves also helping to ensure trust by brokering between the interests of consumers and suppliers through transparent billing and service management mechanisms.

This belief in service delivery is one of the reasons I believe that the notion of ‘private clouds’ is an oxymoron – I found this hoary subject raised again on a Joe McKendrick post after a discussion on ebizQ – even without the central point about the obvious loss of economies of scale; essentially  the requirement to provide a whole business enablement fabric to facilitate cross organisational service ecosystems – initially for SaaS but increasingly for organisational collaboration and specialisation – is just one of the reasons I believe that ‘Private Clouds’ are really just evolutions of on-premise architecture patterns – with all of the costs and complexity retained – and thus purely marketecture.  When decreasing transaction costs are enabling much greater cross organisational value chains the benefits of a public service delivery platform are immense, enabling organisations to both scale and evolve their operations more easily whilst also providing all of the business support they need to offer and consume business services in extended value chains.  Whilst some people may think that this is a pretty future-oriented reason to not like the notion of private clouds, for completeness I will also say that to me  – in the sense of customer owned infrastructures – they are an anachronism; again this is just an extension of existing models (for good or ill) and nothing to do with ‘cloud’.  It is only the fact that most protagonists of such models are vendors with very low level maturity offerings like packaged infrastructure and/or middleware solutions that makes it viable, since the complexity of delivering true private SDP offerings would be too great (not to mention ridiculously wasteful).  In my view ‘private clouds’ in the sense of end organisation deployment is just building a new internal infrastructure (whether self managed or via a service company) sort of like the one you already already have but with a whole bunch of expensive new hardware and software (so 90% of the expense but only 10% of the benefits). 

To temper this stance I do believe that there is a more subtle, viable version of ‘privacy’ that will be supported by ‘real’ service delivery platforms over time – that of having a logically private area of a public SDP to support an organisational context (so a cohesive collection of branded services, information and partner integrations – or what I’ve always called ‘virtual private platforms’).  This differs greatly from the ‘literally’ private clouds that many organisations are positioning as a mechanism to extend the life of traditional hardware, middleware or managed service offerings – the ability of service delivery platforms to rapidly instantiate ‘virtual’ private platforms will be a core competency and give the appearance and benefits of privacy whilst also maintaining the transformational benefits of leveraging the cloud in the first place.  To me literally ‘private clouds’ on an organisations own infrastructure – with all of their capital expense, complexity of operation, high running costs and ongoing drag on agility – only exist in the minds of software and service companies looking to extend out their traditional businesses for as long as possible. 

Follow

Get every new post delivered to your Inbox.

Join 172 other followers