Industrialised Service Delivery Redux II

In my previous post I discussed the way in which our increasingly sophisticated use of the Web is creating an unstoppable wave of change in the global business environment.  This resulting acceleration of change and expectation will require unprecedented organisational speed and adaptability whilst simultaneously driving globalisation and consumerisation of business.  I discussed my belief that companies will be forced to reform as a portfolio of systematically designed components with clear outcomes and how this kind of thinking changes the relationship between a business capability and its IT support.  In particular I discussed the need to create industrialised Service Delivery Platforms which vastly increase the speed, reliability and cost effectiveness of delivering service realisations. 

In this post I’ll to move into the second part of the story where I’ll look more specifically at how we can realise the industrialisation of service delivery through the creation of an SDP.

Industrialisation 101

There has been a great deal written about industrialisation over the last few years and most of this literature has focused on IT infrastructure (i.e. hardware) where components and techniques are more commoditised.  As an example many of my Japanese colleagues have spent decades working with leaders in the automotive industry and experienced firsthand the techniques and processes used in zero defect manufacturing and the application of lean principles. Sharing this same mindset around reliabliity, zero defect and technology commoditisation they created a process for delivering reliable and guaranteed outcomes through pre-integration and testing of combinations of hardware and software.  This kind of infrastructure industrialisation enables much higher success rates whilst simultaneously reducing the costs and lead times of implementation. 

In order to explore this a little further and to set some context, let’s Just think for a moment about the way in which IT has traditionally served its business customers.  

We can see that generally speaking we are set a problem to solve and we then take a list of products selected by the customer – or often by one of our architects applying personal preference – and we try to integrate them together on the customers site, at the customers risk and at the customers expense. The problem is that we may never have used this particular combination of hardware, operating systems and middleware before – a problem that worsens exponentially as we increase the complexity of the solution, by the way – and so there are often glitches in their integration, it’s unclear how to manage them and there can’t be any guarantees about how they will perform when the whole thing is finally working. As a result projects take longer than they should – because much has to be learned from scratch every time – they cost a lot more than they should – because there are longer lead times to get things integrated, to get them working and then to get them into management – and, most damningly, they are often unreliable as there can be no guarantees that the combination will continue to work and there is learning needed to understand how to keep them up and running.

The idea of infrastructure industrialisation, however, helps us to concentrate on the technical capability required – do you want a Java application server? Well here it is, pre-integrated on known combinations of hardware and software and with manageability built in but – most importantly – tested to destruction with reference applications so that we can place some guarantees around the way this combination will perform in production.  As an example, 60% of the time taken within Fujitsu’s industrialisation process is in testing.  The whole idea of industrialisation is to transfer the risk to the provider – whether an internal IT department or an external provider – so that we are able to produce consistent results with standardised form and function, leading to quicker, more cost effective and reliable solutions for our customers.

Now such industrialisation has slowly been been maturing over the last few years but – as I stated at the beginning – has largely concentrated on infrastructure templating – hardware, operating systems and middleware combined and ready to receive applications.  Recent advances in virtualisation are also accelerating the commoditisation and industrialisation of IT infrastructure by making this templating process easier and more flexible than ever before.  Such industrialisation provides us with more reliable technology but does not address the ways in which we can realise higher level business value more rapidly and reliably.  The next (and more complex) challenge, therefore, is to take these same principles and apply them to the broader area of business service realisation and delivery.  The question is how we can do this?

Industrialisation From Top to Bottom

Well the first thing to do is understand how you are going to get from your expression of intent – i.e. the capability definitions I discussed in my previous post that abstract us away from implementation concerns – through to a running set of services that realise this capability on an industrialised Service Delivery Platform. This is a critical concern since If you don’t understand your end to end process then you can’t industrialise it through templating, transformation and automation.

In this context we can look at our capability definitions and map concepts in the business architecture model down to classifications in the service model.  Capabilities map to concrete services, macro processes map to orchestrations, people tasks map to workflows, top level metrics become SLAs to be managed etc. The service model essentially bridges the gap between the expression of intent described by the target business architecture and the physical reality of assets needed to execute within the technology environment.

From here we broadly need to understand how each of our service types will be realised in the physical environment – so for instance we need a physical host to receive and execute each type of service, we need to understand how SLAs are provisioned so that we can monitor them etc. etc.

Basically the concern at this stage is to understand the end to end process through which we will transform the data that we capture at each stage of the process into ever more concrete terms – all the way from logical expressions of intent through greater information about the messages, service levels and type of implementation required, through to a whole set of assets that are physically deployed and executing on the physical service platform, thus realising the intent.

The core aim of this process must be to maximise both standardisation of approach and automation at each stage to ensure repeatability and reliability of outcome – essentially our aim in this process is to give business capability owners much greater reliability and rapidity of outcome as they look to realise business value.  We essentially want to give guarantees that we can not only realise functionality rapidly but also that these realisations will execute reliably and at low cost.  In addition we must also ensure that the linkage between each level of abstraction remains in place so that information about running physical services can be used to judge the performance of the capability that they realise, maximising the levers of change available to the organisation by putting them in control of the facts and allowing them to ‘know sooner’ what is actually happening.

Having an end to end view of this process essentially creates the rough outline of the production line that needs to be created to realise value – it gives us a feel for the overall requirements.  Unfortunately, however, that’s the nice bit, the kind of bit that I like to do. Whilst we need to understand broadly how we envisage an end to end capability realisation process working, the real work is in the nasty bit – when it comes to industrialisation work has to start at the bottom.

Industrialisation from Bottom to Top

If you imagine the creation of a production line for any kind of physical good they obviously have to be designed to optimise the creation of the end product. Every little conveyer belt or twisty robot arm has to be calibrated to nudge or weld the item in exactly the same spot to achieve repeatability of outcome. In the same way any attempt to industrialise the process of capability realisation has to start at the bottom with a consideration of the environment within which the final physical assets will execute and of how to create assets optimised for this environment as efficiently as possible. I use a simple ‘industrialisation pyramid’ to visualise this concept, since increasingly specialised and high value automation and industrialisation needs to be built on broader and more generic industrialised foundations. In reality the process is actually highly iterative as you need to continually be recalibrating both up and down the hierarchy to ensure that the process is both efficient and realises the expressed intent but for the sake of simplicity you can assume that we just build this up from the bottom.

So let’s start at the bottom with the core infrastructure technologies – what are the physical hosts that are required to support service execution? What physical assets will services need to create in order to execute on top of them? How does each host combine together to provide the necessary broad infrastructure and what quality of service guarantees can we put around each kind of host? Slightly more broadly, how will we manage each of the infrastructure assets? This stage requires a broad range of activity not just to standardise and templatise the hosts themselves but also to aggregate them into a platform and to create all of the information standards and process that deployed services will need to conform to so that we can find, provision, run and manage them successfully.

Moving up the pyramid we can now start to think in more conceptual terms about the reference architecture that we want to impose – the service classifications we want to use, the patterns and practices we want to impose on the realisation of each type, and more specifically the development practices.  Importantly we need to be clear about how these service classifications map seamlessly onto the infrastructure hosting templates and lower level management standards to ensure that our patterns and practices are optimised – its only in this way that we can guarantee outcomes by streamlining the realisation and asset creation process. Gradually through this definition activity we begin to build up a metamodel of the types of assets that need to be created as we move from the conceptual to the physical and the links and transformations between them. This is absolutely key as it enables us to move to the next level – which I call automating the “means of production”.

This level becomes the production line that pushes us reliably and repeatably from capability definition through to physical realisation. The metamodel we built up in the previous tier helps us to define domain specific languages that simplify the process of generating the final output, allowing the capture of data about each asset and the background generation of code that conforms to our preferred classification structure, architectural patterns and development practices. These DSLs can then be pulled together into “factories” specialised to the realisation of each type of asset, with each DSL representing a different viewpoint for the particular capability in hand.  Individual factories can then be aggregated into a ‘capability realisation factory’ that drives the end to end process.  As I stated in my previous post the whole factory and DSL space is mildly controversial at the moment with Microsoft advocating explicit DSL and factory technologies and others continuing to work towards MDA or flexible open source alternatives.  It is suffice to say in this context that the approaches I’m advocating are possible via either model – a subject I might return to actually with some examples of each (for an excellent consideration of this whole area consult Martin Fowler’s great coverage, btw).

The final level of this pyramid is to actually start taking the capability realisation factories and tailoring them for the creation of industry specific offerings – perhaps a whole set of ‘factories’ around banking, retail or travel capabilities. From my perspective this is the furthest out and may actually not come to pass; despite Jack Greenfield’s compelling arguments I feel that the rise of SOA and SaaS will obviate the need to generate the same application many times by allowing the composing of solutions from shared utilities.  I feel that the idea of an application or service specific factory assumes a continuation of IT oversupply through many deployments; as a result I feel that the key issue at stake in the industrialisation arena is actually that of democratising access to the means of capability production by giving people the tools to create new value rapidly and reliably.  As a result I feel that improving the reliability and repeatability of capability realisation across the board is more critical than a focus on any particular industry. (This may change in future with demand, however, and one potential area of interest is industry specific composition factories rather than industry specific application generation factories). 

Delivering Industrialised Services

So we come at last to a picture that demonstrates how the various components of our approach come together from a high level process perspective.

Across the top we have our service factory. We start on the left hand side with capability modelling, capturing the metadata that describes the capability and what it is meant to do. In this context we can use a domain specific language that allows us to model capabilities explicitly within the tooling. Our aim is then to use the metadata captured about a capability to realise it as one or more services. In this context information from the metamodel is transformed into an initial version of the service before we use a service domain language to add further detail about contracts, messages and service levels. It is important to note that at this point, however, the service is still abstract – we have not bound it to any particular realisation strategy. Once we have designed the service in the abstract we can then choose an implementation strategy – example classifications could be interaction services for Uis, workflow services for people tasks, process services for service orchestrations, domain services for services that manage and manipulate data and integration services that allow adaptation and integration with legacy or external systems.

Once we have chosen a realisation strategy all of the metadata captured about the service is used to generate a partially populated realisation of the chosen type – in this context we anticipate having a factory for each kind of service that will control the patterns and practices used and provide guidance in context to the developer.

Once we have designed our services we now want to be able to design a virtual deployment environment for them based wholly on industrialised infrastructure templates. In this view we can configure and soft test the resources required to run our services before generating provisioning information that can be used to create the virtual environment needed to host the services.

In the service platform the provisioning information can be used to create a number of hosting engines, deploy the services into them, provision the infrastructure to run them and then set up the necessary monitoring before finally publishing them into a catalogue. The Service Platform therefore consists of a number of specialised infrastructure hosts supporting runtime execution, along with runtime services that provide – for example – provisioning and eventing support.

The final component of the platform is what I call a ‘service wrap’. This is an implementation of the ITSM disciplines tailored for our environment. In this context you will find the catalogue, service management, reporting and metering capabilities that are needed to manage the services at runtime (this is again a subset to make a point). In this space the service catalogue will bring together service metadata, reports about performance and usage plus subscription and onboarding processes.  Most importantly there is a strong link between the capabilities originally required and the services used to realise them, since both are linked in the catalogue to support business performance management. In this context we can see a feedback loop from the service wrap which enables capability owners to make decisions about effectiveness and rework their capabilities appropriately.

Summary

In this second post of three I have demonstrated how we can use the increasing power of abstraction delivered by service-orientation to drive the industrialisation of capability realisation.  Despite current initiatives broadly targeting the infrastructure space I have discussed my belief that full industrialisation across the infrastructure, applications, service and business domains requires the creation and consistent application of known patterns, processes, infrastructures and skills to increase repeatability and reliability. We might sacrifice some flexibility in technology choice or systems design but increasing commoditisation of technology makes this far less important than cost effectiveness and reliability. It’s particularly important to realise that when industrialising you need to understand your end to end process and then do the nasty bit – bottom up in excruciating detail.

So in the third and final post on this subject I’m going to look a little bit at futures and how the creation of standardised and commoditised service delivery platforms will affect the industry more broadly – essentially as technology becomes about access rather than ownership so we will see the rise of global service delivery platforms that support capability realisation and execution on behalf of many organisations.

:: :: :: :: :: :: :: :: :: :: ::

Service Delivery Platforms peek from behind the Cloud

Quick note having just read Dion Hincliffe’s article on Google App Engine and Amazon Web Services (their respective cloud computing infrastructures).  These platforms are early and very basic instances of the ’service delivery platforms’ I’ve talked about for the past few years (for an example see a presentation I gave at the MS SOA & BPM conference last year).  As I discussed during this talk, I use the term ’service delivery platform’ in place of ‘platform as a service’ or ‘cloud computing’ since in order to really support viable and fully rounded service delivery you need to provide far more than a ‘platform’ or ‘infrastructure’ – a topic I’m going to return to now that this area has been popularised by Google and I won’t seem like (such) a lunatic, lol.  More broadly, however, I’ve discussed a number of times why I feel that economics and technology commoditisation will drive people down this route eventually and I’m really excited now to see a number of competitors emerging in this space (with Amazon, Google and Salesforce now competing with a number of smaller startups (with more to follow)).  I know I’ve said this before but people are really going to have to start deciding what business they are in – if you work in an end user organisation then you need to recognise that and start treating IT like a utility rather than a differentiator; if you’re an IT service company, however, you’d better work out whether you want to be focused on relationship brokering and consulting, utility computing platform provision or  SaaS/BPU service offers, since the economics of each are very different (you may in fact want to play in all three but if so you’d best disaggregate your company and run them autonomously under your umbrella brand – or you’ll make a complete hash of them all).

One of the interesting things for me is whether a middle-ground model will emerge that enables enterprises to take advantage of the industrialisation and economies of scale available to service delivery platform vendors whilst also enabling the deployment of ‘edge’ infrastructures to customer specific environments.  In this context we may see locally deployed ‘chunks’ of the central service created for those organisations that are large enough or who have specific privacy or trust issues (still coordinated with and managed from the centre, however).  Whether such a model has a long term future is an interesting (but as yet undecided) question to my mind but in the short to medium term those SDP vendors who were able to deliver such a transitional solution would provide a compelling migration path to enterprises who are nervous about the implications of a wholesale shift to the cloud.

:: :: :: :: :: :: :: :: :: :: ::

Relationships vs Transactions

Background

Got a comment from one of my colleagues – a chap called Martin Abbot – by email a couple of days ago with respect to my last post about institutional innovation.  Whilst I’ve been planning to find some time to write a follow up that links these ideas through to SOA and SaaS I thought it might be worth just posting the question and my email response.  It’s not a ‘proper’ blog post – and so is a bit messy – but the questions and my rushed answer are probably worth sharing.

The Question

“Is the idea of moving away from transactional relationships to ones that are mutually beneficial not a little at odds with SaaS and SOA generally? Both of these seem to support the commoditization of resources and are therefore inherently transactional in nature.”

The Answer

Well I guess that there are a few things:

Lots of stuff tends towards commoditisation and so in many ways the argument Martin makes is understandable.  Even based purely on the use of ‘commodity’ services, however, I guess you could make the following arguments:

• Commodity doesn’t necessarily mean wholly standardised.  In the future we’re going to need to support mass customisation of services to support our customers businesses.  In a SaaS/SOA environment this basically requires us to have many customisation points built into the software that support both multi-tenancy and customisation.  The key here is to build the software for customisation in a few key dimensions from the ground up rather than rely on your ability to bastardise it later.  To be able to understand the 20% key dimensions that need to be customisable, however, you need a pretty deep understanding of your consumers (I realise that this point is only tenuously related but I need to establish it first…)
• The second point is that even if you are only delivering commodity services there is always the opportunity to a) understand your customers needs better and b) deliver services that are more appropriate to them through your improved contextual understanding.  This requires collaborating partners to share information much more freely in order to get the best service.  When you combine this with the ability to mass-customise services you can see that both the provider and consumer get best advantage when they each fully understand the capabilities and aspirations of the other.  This depth of relationship requires a degree of trust that goes above and beyond a traditional zero-sum approach, however.
• From a provider perspective you also need to work closely with your partners to understand the effectiveness (or not of your services) in order to be able to shorten your feedback loop and accelerate the improvement of your capabilities.  Again this is a win-win scenario potentially as both parties get greater value.

All of this is fine from the perspective of thinking of two parties as ‘customer’ and ‘provider’ when the main goal is the consumption of services on a transactional basis.  i.e. there are still some low level advantages to both parties of creating a closer relationship.

Beyond this however is the real value.  If you build trust across organisations that offer some part of the value chain then the more important question becomes how you can leverage your capabilities together in order to improve existing offerings or – perhaps more importantly – create new services or offerings to existing or new markets.  The key here is that each service provider may have a fairly conventional view of what services they offer and who they serve but when each comes together to look at how they collaborate to deliver  value and how each others services could be used to approach their customers then a new perspective opens up.  Once this perspective is opened they can then begin to consider how each would need to optimise their services in order to create these new markets and this in turn can help them to consider how these ideas impact (and potentially improve) their ‘traditional’ offerings.  When you broaden this from two parties into everyone who performs any value adding activity within and across value webs you can see that the opportunities for collaboration and new value creation rise exponentially.

This is also applicable when you think of ‘customers’ in the traditional sense rather than groups of suppliers working together;  customers and their providers exist in a value-web and therefore by forging strong relationships and inviting suppliers to help improve the end product the overall value for everyone is greatly enhanced.  This is also a two-way dialogue since providers can use Web 2.0 techniques to bring their consumers into the service creation process and thereby become co-creators of the products and services that the company offers – an ideal way to balance push and pull models.

Even if we think about products and services that tend towards commoditisation – and many things do – we can see that the reality is that such commoditisation is inevitable and accelerating; sharing information with trusted partners – whilst it may hasten the commoditisation of certain services – more than offsets this by opening up far grander opportunities to utilise these services in new ways.  Furthermore if your services are subject to commoditisation then they are also subject to economies of scale and leveraging partners to find new ways of using these commoditised services to build new markets is a much smarter move than withdrawing into yourself and becoming obsessed with efficiency and cost reduction. 

To paraphrase Bill Joy, there are far more smart people outside your organisation than in it and maximising your ability to innovate by mutually leveraging this smartness is increasingly going to be a necessary capability.  As I stated in my original post, I agree that there are many things to be worked out in order to move people into this new way of working but I believe that those who make the transition will create and reap significantly greater value than those who do not.

Conclusion

My anticipated next post was to actually look at how service delivery platforms and SOA could help to accelerate the types of relationships that I discussed.  I’m still aiming to write that post but decided that there was value in exposing this information in the interim as a) it is related and adds to the discussion and b) given my excessive commitments at the moment it was easier than actually finding the time to write the post I need to at this point in time

:: :: :: :: :: :: :: :: :: :: ::

Stability, Relationships and Institutional Innovation as Pillars of Success

Just read a few pieces from John Hagel – one of my favourite writers – whilst catching up on my blog backlog.  Essentially two posts caught my attention, one describing some areas of concentration for the coming year and another exploring the concept of institutional innovation.  Both of these posts resonated strongly with me and I felt that elements of both were strongly – even inextricably – linked together.

The two main points from the first post that caught my attention were around the value of stability in a changing world and on the decreasing value of transactional behaviour.  These ideas were reinforced by the second post about institutional innovation as I believe that such transformational innovation requires us to recognise the need for stability whilst simultaneously downplaying transactional behaviour in favour of mutually beneficial relationships.

I’ll explore this a little further.

The importance of internal stability

I’ve discussed many times how I believe that in order to be successful in a rapidly changing world we need to look at the structural properties of our organisations before considering how they work.  In this context understanding the capabilities that we need to be successful – along with the metrics that they need to support – is a key lever to greater organisational adaptability.  This is so since these abstractions allow us to concentrate on what we need to achieve whilst burying the detail of how we achieve it to prevent ourselves from being overwhelmed and therefore powerless to act decisively.  Equally importantly, however, a concentration on structural components and their relationships allows us to escape from tightly coupled process-oriented thinking and move to an organisational style that is more loosely coupled and output-driven. 

In this discussion, however, the key characteristic of interest in a structural view is that of stability – essentially a structural, capability based view of an organisation is far more stable than a dynamic, process-oriented one.  This is because the things that we do broadly remain static over time whilst the way in which we do them can be impacted hugely by influences such as technology advances, economic pressures or the talent pool we have at our disposal.

As increasingly rapid change impacts our organisations and forces us to rethink how we do things, the value of understanding explicitly what we do cannot be overstated.  Chaos ensues when people have no fixed reference points around which to manage change; a stable view of intent, however, allows us to have the benefit of both worlds – a stable understanding of things that need to be achieved and therefore a clear field of operations in which to implement change in a decisive and systematic fashion. 

The importance of external stability

Taking this thinking to the next level, increasing pressures on organisations will increasingly force them to look at the set of capabilities that they have and decide which ones represent their core areas of expertise (it is worth stating here that this is another, related reason to take a systematic view of the capabilities an organisation needs but that is assumed in this post).  This choice will increasingly be driven by economic forces and so – to use another of John’s broad ideas – organisations will need to decide whether they major on customer relationships, infrastructure or innovation.  Making this decision will be a further expression of stability at the macro level, however, since our organisation will now be signaling to its wider ecosystem that it has capabilities of a particular nature that can be leveraged and also that it will be looking for partners to provide other elements of the value chain on its behalf.  At the top level well positioned service providers can therefore become points of stability in the changing landscape of an industry, enhancing both their own positions but also providing reference points to new entrants trying to excel in a different section of the value chain. 

The futility of transactional behaviour

One of the key issues that emerges as we begin to seek complementary services from partners is the nature of the relationships we wish to build.  Current transactional based thinking would tend to treat partners as ’suppliers’ held at arms length and squeezed for the minimum costs sustainable.  This approach ignores the far greater benefits to be accessed by building and leveraging these relationships to achieve profitable and sustainable growth on both sides and represent a zero sum game in place of efforts to mutually grow the available opportunities.  Many kinds of emerging relationships – whether those lauded by social networking advocates mentioned by John or those enabled by increasing B2B information exchanges – are merely transactional in nature and not relationships in the true sense of the word.  If we are to maximise our opportunities and accelerate the improvement of our chosen capabilities then we must build deeper relationships in place of impersonal transactions in order to leverage complementary expertise and expand our influence and opportunities into new markets.

The increasing importance of deepening relationships

Increasingly we are seeing a shift away from consumer-supplier relationships where one side owns the power towards a situation in which different organisations are composing value for customers out of their complementary capabilities.  In this context relationships are of far greater value than the lowest possible cost because we now need to leverage our combined talents to improve all of our services and by extension the overall result as experienced by the end customer.  We are therefore going to be looking to leverage our different perspectives and expertise in order to look at the overall value being created and to realign our individual services to improve the output of our joint activities

Two points in particular stood out for me in John’s post in the context of relationships:

• Deep relationships become increasingly valuable in times of widespread change;
• in fact, deep relationships are essential to capture the full value of weak ties.

Again one of the broader questions around the need for stability is what and who we can count on to help us make sense of the changes that are happening around us.  In this context the deeper our relationships with our partners the more we can both consider them a point of stability and support as we deal with change but also the broader range of perspectives and insight we have in making sense of change and in deciding how to react most effectively. 

In the second case it is worth highlighting the fact that deep relationships do not automatically infer tight integration.  In building relationships with partners who will offer their complementary capabilities to us we need to ensure that we deliver the loosest possible coupling between our respective services.  Counter-intuitively perhaps this is not because we wish to return to a view of our partner as a supplier to be easily swapped out but rather that we wish to give them the maximum scope for innovation in the services that they deliver to us.  In this context – again counter-intuitively perhaps – we require much deeper relationships and trust to be in place before we will consent to the loose coupling required to maximise innovation – the natural tendency when we unbundle capability to another is to want to understand and control the way in which things are done on our behalf.  This is limiting behaviour since we fail to take advantage of the complementary expertise of the specialised partner and instead continue to project our inadequacies onto their delivery and constrain their ability to deliver the best possible service on our behalf.  Breaking this habit requires deep and trusting relationships, however, since the more interdependent we become the more we each depend upon the other for mutual success and therefore the more we tend to want to feel in control.

Leveraging stability and deep relationships to realise institutional innovation

These ideas around the importance of stability and the inadequacy of transaction thinking lead onto a consideration of John’s other post around institutional innovation.  Developing a stable view of your organisation, zeroing in on your key capabilities and then concentrating on changing the nature of your relationships with partners can open the door to the benefits of institutional innovation:

Stability:  We know what we do and what our partners do and so can successfully build relationships around these points of stability.  We have a context for innovation across organisational boundaries.

Modularity:  A shift to a stable view also drives a more modular approach.  We can therefore minimise coupling and maximise innovation opportunities.  We therefore have a context for distributed and deconflicted innovation implementation.

Relationships:  Developing deep, positive sum game relationships enables us to jointly seek mutual advantage with our partners through leveraging our diversity.  We therefore have a context for longer term capability development and organisational growth.

Essentially as we and our partners become more dependent on each other to realise value so we also become dependent on each other to realise such value in the most effective way possible.  In this context innovation is no longer something that happens within the bounds of your organisation but rather is most potent at the intersection of your capabilities with those of your partners. 

As John points out such innovation transcends current practices around ‘open innovation’ and moves from point attempts to leverage occasional 3rd party expertise to inform our own innovation towards sustained and systematic examination of innnovation opportunities across our partner ecosystem.  This broader approach requires us to continually leverage the diverse experience, expertise and perspective of our wider ecosystem to look for mutual advantage, a subtle but important difference.  Essentially such innovation recognises the increasingly symbiotic nature of partnership and the huge opportunities to create breakthrough innovation through diversity  Such innovation may come in the form of improvements to individual capabilities as a result of partner feedback, in improvements in the functioning of the overall value web or in insights regarding new joint offering or market opportunities. 

Summary

This is an emerging area that requires us to rethink the way in which we deal with partners, the way in which we locate our people and the tools and technology that we can use to support distributed co-creation and innovation.  It may be that some of the tools are already here – I’ve written about using Web 2.0 techniques to leverage talent, for example – but our understanding of the practices and processes that will enable us to really exploit these opportunities is still limited.  Given the rich seam of benefit to be mined through broader, more collaborative innovation, however we all have a duty to promote and develop these ideas as quickly as possible.

:: :: :: :: :: :: :: :: :: :: ::

Enterprise Architecture is yesterday’s news

Amazed to find this post that suggests that Enterprise Architecture is the next big thing.  I thought that EA was a late 90s, early 00s silver bullet and that we were now all moving on to a concentration on fractal business architectures supported by federated IT.  Now I realise that judicious EA can encompass this view – indeed that it should – but realistically EA as a term was hijacked by geeks who used it to enforce technical standards and who gibbered at their business colleagues in a threatening way about governance and compliance whilst creating impenetrable mountains of useless documentation that was out of date as soon as it was created (and made no sense to business people or developers). 

Worse than that most EA practitioners genuinely believe that they can document the whole organisation top down in a complex chain of dependencies – a horrible fallacy in a world of federation – and then micro-govern people by putting cumbersome processes in the way of getting anything done (and given the difficulty (nay – impossibility) of doing this they end up obsessing on forcing shared and inappropriate IT on people instead of supporting business improvement). 

Taking it all a bit further it’s also unfortunate that most EA efforts top out at business processes (rather than capabilities) and so are a pure expression of how things get done rather than a sensible framework for enabling decisions about what should be done (I enjoyed Steve Jones’s post about this very issue).

Before I get flamed to the extent that I am consumed by a terrible conflagration I have to say a couple of other things; are the aims of EA crazy (i.e. to understand and govern the organisation)?  Umm, no.  Should we still be seeking to do this?  Umm, yes.  Does the kind of top down, all encompassing approach taken by most EA efforts actually work?  Umm, no.  By saying that EA is no longer useful I’m basically saying that the concept is absolutely right, it is more necessary than ever in a world of services – indeed services could be the missing abstraction that finally fill the gap between intent and execution – but that initial efforts in this direction didn’t take sufficient notice (indeed why would they have) of unbundling and federation.  So I believe that EA needs to evolve to help govern the portfolio of capabilities required by an organisation to function and to include some light touch policies that specify how they should work together.  Below this level we start again, essentially with a smaller EA for each capability, recognising federation and unbundling will remove our ability to control every aspect of the way that an organisation works centrally and from top to bottom – EA essentially needs to become explicitly fractal.

Either way I’m not sure that EA is the next big thing, but then again maybe I’ve just dreamt all of this and the future is bright etc.

:: :: :: :: :: :: :: :: :: :: ::

Killer, Gorilla, Guerilla SOA

The Proposition

Just read an interesting post by Jim Webber about a lighter weight technique for SOA implementation that he calls Guerilla SOA.  This spurred a number of comments from people such as Joe McKendrick, Tony Baer and ZapThink’s Jason Bloomberg.  Generally the comments were positive excepting the usual concerns about bottom up implementations leading nowhere.

Guerilla Campaigns Need Direction

As I’ve discussed previously I’m a big fan of thinking broadly but starting small, since with difficult to grasp concepts like SOA we have to rapidly and continually deliver benefit to generate buy in from our business colleagues.  As a result I am intrigued and broadly supportive of the concept of guerilla SOA implementation.  The key thing in my view, however, is to start in the right way by concentrating initially on getting a broad view of the business capabilities needed by the enterprise through a rapid approximation and then use these as a governing framework for the implementation of services within the organisation. I’m a firm believer in mixing systematic design with a dash of chaos – essentially govern the top level business capabilities required by the organisation but then delegate responsibility for the realisation of these downwards (and since this is a fractal process there are a limited number of capabilities at each level and as such the decision making process is made simpler by restricting the number of abstractions that any one group of people need to manage).  As a result we allow capability owners to realise services any way that they want – potentially meaning that many replicated services spring up across the organisation within the control of different capabilities – since the only thing we care about is that we have the ‘right’ capabilities and that these perform to the cost and quality metrics that we require.  In this context sharing of services at lower levels of granularity will be an emergent property caused by attempts to improve the cost effectiveness and quality of capabilities rather than something that is planned for from day one (which requires a deep and complex understanding of how everything in the organisation works from top to bottom – an impossible task despite the best efforts of many current enterprise architecture approaches).  Equally importantly in this context, attempting to force capability owners to share smaller granularity services by looking for ‘reuse’ often damages these business capabilities in subtle ways that are impossible to guess ahead of time – as a result independence with future emergent sharing opportunities based on metrics is a more appropriate mechanism IMHO (which now that I’ve read that sounds a bit like the agile principle of only building what you need – hadn’t really thought about it within a business context like that before – gives a whole different spin to the hackneyed term ‘agile business’).

Executing Your Campaign

Now the question is how this relates to the discussion of guerilla SOA?  I would argue that the notions of business capabilities and guerilla SOA are orthogonal and complementary.  Essentially you map out the broad ’shape’ of the organisation (your campaign goals) using business capabilities – consider this the horizontal dimension – but then concentrate on federated projects (guerilla operations) that maximise value in the improvement of these capabilities and fill them out using agile implementation styles, essentially rapidly implementing elements of capabilities with focused projects – consider this the vertical dimension.  The reality is that no business design (or indeed guerilla campaign) can ever be correct and pre-planned from top to bottom at any point in time and so the key is to rapidly approximate the systematic capabilities needed by the organisation and then get on with implementing value in a stepwise fashion.  Your governance process will need to be able to cope with changes – e.g. versioning of capabilities, business messages and everything else – as your transition to SOA proceeds but there is no way to nail all of this up front and for ever and so you’d better get used to that and move on.  Essentially anyone who comes to you and suggests a top-down process re-engineering exercise should be politely shown the door, since such exercises take years, are always out of date and never actually deliver any sustainable value.  Let’s call this top down approach “Killer SOA” as it is guaranteed to go nowhere and will quickly lead to a morass of paperwork that scuppers your attempts at SOA.  Conversely however just pitching in like a gorilla without any context will make you look busy and actually deliver some stuff but it won’t really add up to anything that makes a significant difference to your organisation – the sum of the parts will really be just the parts (and it’ll be more complex and expensive to deliver, operate and manage).  Let’s call this bottom up approach “Gorilla SOA” as we can imagine it as a chase for the next banana without any concern for what such plundering might do to the environment that sustains us.  To me Guerilla SOA takes the best approach possible, relying on a combination of a broad plan and lots of loosely coordinated and federated cells with the power to implement in a situationally appropriate way.  

Enable Guerillas to Improvise in Realising Goals

This approach to business capability realisation is also applicable in the technology space as well.  I am glad to see Jim stating (and an increasing number of other people realising) that technology as architecture (e.g. ESBs/application servers/whatever) is both meaningless and dangerous.  As I’ve discussed in the past trying to force centralised architectural approaches onto an essentially heterogeneous business environment (heterogeneous from a requirements perspective (e.g. QoS or cost requirements) as well as a technical one) leads to an environment that is fit for nobody.  Architectural approaches should concentrate on the space between capabilities (i.e. standards that support interoperability (or an overall loose coordination of cells)) and on the governance of the required outputs (i.e. capability metricisation and performance management (or realisation of strategic operations)) and enable capability owners (and guerillas) to maximise existing assets or new approaches that best map to their goals and aspirations rather than on trying to use technology as an excuse for architecture (I have to also admit to having a particular distaste for ‘ESBs’ which in most incarnations are just hub-and-spoke message hubs with proprietary transport which undermine the very notions of SOA – i.e. federation, standardisation and interoperability – but I guess that’s another post, grrr ).

It’s the Difference Between Killer, Gorilla and Guerilla that Matters

In the broader context the question is whether Jim’s comments reflect an anarchic and directionless dead-end of random service implementation or a dynamic and manageable way to break the SOA deadlock within organisations.  To me the question you have to ask yourself is whether you are a Killer that drowns your organisation in paperwork, a Gorilla who just wades into SOA in an elemental fashion with little thought beyond the moment or whether you are a Guerilla who is part of a larger group of loosely federated fighters striving to realise a shared vision. 

P.S.

I realise that many people refer to ‘Guerilla SOA’ as ‘Middle Out’ (see Nick Malik for an example).  To me the two terms describe the same thing but whereas ‘Guerilla’ makes me think of glamour, adventure and action ‘Middle Out’ makes me think of the male aging process and brings to mind increasing flabbiness around the midriff and balding.  Just sounds less exciting doesn’t it, especially when we’re trying to convince people to follow us into action, LOL

:: :: :: :: :: :: :: :: :: :: ::

SCA = Scary Component Architecture?

Only just catching up with SCA/SDO as part of some work with a current client and have to say that there seems to be a great deal of confusion around these essentially useful specs.  First of all read the threads over at Joe McKendrick’s blog where a conversation raged for a while over the utility of SCA but then also came across this blog post by Dana Gardner over at FASTForward complaining about Microsoft’s unwillingness to play ball.  This latter post confused me deeply.  Dana (and others that he references) seem disturbed by the fact that Microsoft are not involved in this effort and that this represents a blow to ‘SOA interoperability’ and an unwillingness from Microsoft to be open.  This seems like a strange statement to me as from what I can glean from the specifications for SCA and SDO – and from great resources such as those provided by Dave Chappell – these proposals seem to be more about how you create components that sit within a particular realisation environment – realistically Java ones given the spread of vendors involved – and they seem to say nothing about interoperability per se.  Interoperability across realisation technologies (even across specific containers in fact) is left – very sensibly in my view – to existing web service standards.  So far so good in my view as the Java community have badly needed some support in simplifying the creation of lighter weight services of the sort needed in a federated world, since Java EE’s component roots are based on a previous generation of architectural model, IMHO, and as a result it has become way too complex.  Suggestions that Microsoft are not ‘open’ however on the basis of the fact that they only provide support for interoperability standards seems perverse to me.  Whether you are using SCA to compose applications that will realise services within a specific Java vendor’s environment, using WCF within the Microsoft world – or indeed ignoring all of these and resorting to simpler, more dynamic environments such as Ruby – surely having a range of programming models and platforms competing to realise services most efficiently is a healthier model than trying to force them to create service realisations in the same way?

The suggestion that we would have a common programming model that transcends implementation language, however, (as seems to be implied by Dana’s post and indeed others that I’ve read) seems a bit oxymoronic to me as surely our existing XML standards (of whichever flavour you prefer) take care of the multi-language aspect of the problem space, leaving composition as the main issue to be addressed (since BPEL only partially addresses this space).  I’m all for interoperability as a way of enabling competition in the provision of implementations but less ecstatic about the idea of trying to arrive at a converged programming model with a confusing mix of service and implementation metadata.  To be honest I feel that trying to mix the interoperability and implementation spaces by pushing a converged realisation model onto vendors introduces unnecessary complexity and so would not be a good approach to solving the issues around service composition.  Trying to talk SCA up as a way of enabling interoperability across implementation technologies (which again I have to stress is not something I believe the spec authors are doing) is a shame as it confuses an area around which we absolutely need more research and guidance, namely specific ways of modelling composite services – whether UML-based via the OMGs recent call for a services profile, as part of the STP project within Eclipse or via DSLs such as those supported by Microsoft. In each case, however, I feel that these should be at a level of abstraction above the individual technologies being composed since surfacing this information makes consumption a much more complex proposition by putting the onus on the consumer to take responsibility for integration rather than forcing the provider to expose their services using existing standards.

I guess I’m also influenced by the fact that I lived through the CORBA period where we tried to implement a standard programming model and – realistically – failed.  When I compare the subsequent success that the industry has had by concentrating on simplicity and interoperability suggestions of conformity of approach across realisation  technologies feels – to me at least – like a backward step out of touch with the current zeitgeist; currently we’re increasingly looking at simple, open, web based technologies that are ‘good enough’ and rail against the complexity of WS-*.  Such pragmatism has unleashed a wave of innovation from companies both large and small and anything that could introduce constraints and confusion to this model by trying to develop an uber-model of implementation feels to me like a bad idea. One of the great drivers of recent success has been federation – moving responsibility for service enablement to the endpoints (i.e. providers) to enable consistent and simplified consumption of these by consumers.  Trying to deliver a cross-language rather than web based composition model feels to me very much like the old school EAI approach that we have only just recovered from, moving information about endpoints back into the consumer space and increasing the complexity of the composition space.  it all makes me feel vaguely like I’ve returned to the mid-90s (not something you want to do if you’re Welsh as it was probably one of the worst periods in the checkered history of our rugby team).  At the end of the day I would rather see standardisation at the ‘what’ level (i.e. interoperability) than at the ‘how’ level (i.e. programming model) since it feels to me that this is the sweet spot for having your cake and eating it – an ability to work simply and consistently with anyone whilst allowing them maximum scope for innovation on your behalf. 

Whilst being uncomfortable with the idea that SCA represents some kind of uber-programming model, however, I do absolutely recognise the need to simplify service realisation in the Java space and therefore am heartily on side with it’s core aims.  I guess I’m more interested in the next level of composition, however – i.e. composition across standards-based services – rather than just on improvements in the way that one or more particular technology stacks supports developers in the creation of services that execute within their environment.

:: :: :: :: :: :: :: :: :: :: ::

Is BPEL4People really for The People by The People?

I was interested to read lately that the long and tortuous process of arriving at some level of human process support within BPEL continues.  Now I have to be honest and say that to me this seems like a really dumb concept.  If you look at service-orientation as a way of abstracting away from issues of implementation and concentrating on outcomes then I really don’t understand how creating an implementation specific extension makes any sense at all (implementation in terms of service realisation type rather than technology in this context).  What other specific kinds of service types are we going to implement?  BPEL4CICS?  BPEL4ESB?  all of which take account of the differences of implementation or interaction model forced by these implementation choices?  Surely any work that needs to be carried out by people is just another service with a contract, service level agreement and cost?  Yes there will need to be a suitable implementation of that service that supports its execution by people but the fact that the task is carried out by a human should surely be transparent to the consumer (which in this case is the BPEL process).  To be honest I have never been particularly keen on the idea of people support within BPEL as I have always viewed it as a standard which provides support for the creation of ‘Macro’ level processes that represent overall value-chains spanning services.  The services themselves will obviously require many specialised implementation methods but this is where service providers will compete and where technology providers will compete in supporting them most efficiently.  The value of BPEL appears to me to be that it allows us to leverage all services in a consistent way and so extensions to support a particular kind of implementation choice seem to lack elegance to me.

Equally importantly in this discussion, however, is the desirability of the kind of process support that BPEL can implement for people.  As I discussed in a previous post, increasing reliance on talent and the collaboration of talent for competitive differentiation means that the kinds of industrial age processes supported by standards like BPEL will be wholly insufficient.  Essentially BPEL implements pre-configured, task driven processes and cannot support the kinds of collaborative, human driven processes that we will increasingly need to leverage the talents that we have both within and across organisations.  As we increasingly automate the rote tasks – ably supported by BPEL – there will be diminishing returns available and certainly little or no competitive differentiation, since by definition anything that can be codified into a consistent transactional process can be copied.  As a result we need to concentrate on how we maximise the effectiveness of our people as they use their talent, discretion and imagination in responding to service requests in appropriate ways.  BPEL4People does not address these needs in any way since it simply treats people as transactional resources who will tick the pre-configured box as part of the wider machine. 

As a result I think that BPEL4People is a pointless exercise since it extends BPEL with unnatural concepts tailored towards a specific kind of service implementation whilst simultaneously failing to present us with a solution to the real problems of people centric processes.  I strongly believe that we need to accept that the issues to be addressed by transactional and tacit processes are very different and that we should therefore embrace these differences rather than try and create an inappropriate uber-model.  In both cases the concepts of service orientation can be used to offer up the resulting implementations in a consistent way - as long as we continue to adhere to interoperability standards in making the services available for consumption.  As an example Keith Harrison-Broninski talks about using services to allow interoperability between transactional processes implemented by BPEL and any supporting tacit processes implemented by Human Interaction Management products.  The key point in all of this, however, is that services will be realised in many different ways, often very specialised to the domain of implementation and I feel that we should recognise this rather than try and mix abstract process control across consistently described service interfaces with the special ways in which those services are implemented.

As a result I believe that BPEL should be left to cover the consistent orchestration of services irrespective of whether the services are implemented by machines or people and that other more promising techniques should be used to realise services appropriate to the way in which people really work.  The fact of the matter is that BPEL4People seems more like BPEL4Vendors to me; they all want to be seen as BPM vendors rather than integration vendors and in so doing appear to be missing the point completely by treating people like the machines they’re used to dealing with.  The irony of rigid process support appropriate to machine integration being touted as ‘for people’ seems as amusing as it is sad.  The shame is that many organisations will follow the lead of these technology companies as they muddy the water for their own ends and will therefore continue trying to squeeze their people into inappropriate processes that fail to deliver the support that people need in the new connected economy.  In particular, inappropriate use of BPEL4People due to the ‘people’ moniker will:

• Fail to support collaboration and talent amplification by setting out processes as a set of rigid tasks passed out to individuals;
• Constrain innovation in service delivery by assuming that processes can be designed and fixed in advance, making the contextual tailoring needed during human effort extremely difficult;
• Fail to deal with the increasingly dynamic reality at the edge of organisations through rigidity of process and thereby put strain on customer interactions by making them unnatural; and
• Destroy morale and productivity by forcing people to work in ways that are alien to them and in so doing become something to waste time working around rather than a valued support tool.

In many cases these issues will lead to the work being pushed out to people with no hope of actual process support, since there will be a recognition that they will work in some unstructured way to complete the broad task stated.  In this context the work disappears into the shadowlands of the unknown until someone helpfully comes back and updates the system (maybe) – not really a good way of tracking where we are with the mass of human effort that goes on within our organisations every day.

As a result the choice would appear to be to constrain people unacceptably or lose control over what they are doing.  Does that constitute BPEL4People?  Not for this person.

:: :: :: :: :: :: :: :: :: ::

SOA, Web 2.0 and the End of Drudgery

The Growth of Services

One of the great boons of the growth of XML has been the way in which it has transformed integration – increasingly we’re seeing information flowing around the Internet as a set of self-describing documents that contain their own context.  Many of the services that process these documents may be automated – like many SOA or Web 2.0 services that we see today – but many will actually be routes to people (like Amazon’s Mechanical Turk – genius).  In this post I wanted to just explore some of the implications of this, since I believe that human-provided services will come to have very different characteristics from those provided by machines. 

The Growth of Drudgery

Over the last twenty years we have seen increasing concentration on the automation of transactional tasks (i.e. things which can be ‘proceduralised’ and repeated).  This has yielded enormous cost and efficiency benefits but we’re increasingly finding ourselves passing those benefits onto consumers.  As a result cost-cutting and efficiency are becoming a way to remain in the game rather than a source of competitive advantage.  In fact many people – most famously Nick Carr – argue that the use of IT to automate rote processes no longer yields any opportunity for differentiation, since all companies have access to the same technical capabilities and the work being automated is itself highly proceduralised and therefore by it’s nature easy to replicate.  Part of the rush to automate and proceduralise, however, has seen organisations place considerable emphasis on ‘workflow’ technologies that attempt to box people into specific tasks within an overall process, with little chance for innovation or creativity.  Such tools generally emphasise control and repeatability over contextual decision making – in a Taylorist mould – and therefore limit the freedom of the people and teams receiving work requests.  As a result we have seen a huge increase in the amount of work that is fixed, detailed and monitored, resulting in what I would call drudgery. 

Talent as the New Competitive Differentiation

In parallel with the realisation that increasing automation is reducing the scope for differentiation in transactional processes we have started to realise that – unsurprisingly – real, hard to recreate differentiation is found in the knowledge and talent of our people.  Most organisations make little or no attempt to recognise and promote the value of talent, however, and in fact encase people within industrial age processes that attempt to control their every response irrespective of context, reducing the scope for them to creatively tackle problems and find solutions whilst simultaneously reducing the adaptability and initiative of the enterprise as a whole.  Despite the continued belief in many companies that ‘the centre’ knows best, we’re actually seeing a movement towards increasing federation; as the world becomes a more uncertain and dynamic place, so the ability to compose the ‘perfect’ plan from the centre and then just get people to execute it breaks down completely.  As a result we need to move decision making to the edges of process boundaries through the coordination of loosely coupled services.  The benefit of this federated approach is that it moves the decisions on how to meet goals much closer to the people that have the knowledge to best decide what needs to be done whilst maintaining governance through concentration on outputs and KPIs in place of tight control of the tasks completed.  This is a major shift in management philosophy, relying as it does on service contracts, trust and a belief in people in place of absolute control of every stage of a process.  The return, however, is that we get more appropriate, adaptable and innovative services that leverage the wealth of human talent available to us in place of smothering it. 

As a result one of the major tasks facing all industry sectors is to yes, continue to automate rote tasks to drive efficiencies but also to realise that real, difficult to replicate competitive advantage will come from the way in which we allow creative people to use information at the edge in providing services both to customers and back into the organisation.  As transactional jobs are increasingly automated, outsourced and commoditised (since it is difficult to create non-replicateable differentiation in these due to the fact that such processes can be easily codified and replicated) we will need to break down our old notions of control and embrace trust based models to support creative, customer relationship or problem resolution processes where real differentiation will be delivered which is difficult to recreate (due to the fact that it is dependent on talent, contextual awareness and experience – three things which cannot be codified).

The Alternative Growth of Extreme Federation

Whilst much of this work has been going on we’ve seen an alternative approach to command and control workflow systems growing on the Internet.  Increasingly complex applications, businesses and social networking sites have sprung up from sets of loosely coupled people and systems with the emphasis much more firmly placed on expectations and trust.  In such extreme federation people concentrate on the required outcomes whilst wholly devolving the way in which those outcomes are achieved to friends, partners or 3rd parties.  In solving particular problems, people come together within newsgroups, on wikis, via blogs or (even – shock) via email, collaborating to solve problems together within their sphere of expertise.  One of the most extreme examples of such community based activity is the open source software movement, where high levels of trust and collaboration yield levels of innovation, creativity and quality that often outperform traditional command and control methods of management within product development companies.  One of the key aspects of this movement, however, is the leveraging of human capital to take advantage of the well of creativity and talent that this represents instead of merely seeing people as drones to effect some part of a pre-planned process.  Many of these changes have resulted in the emergence of the group of concepts known as Web 2.0, with key elements that support people, their collaborations and their access to services in a personalised way.  By supporting such collaboration these technologies are ushering in an age of accelerated innovation, by bringing together many experiences and perspectives and allowing the people involved to interact, argue and learn, building new value faster than ever before.

Human Vs Machine

In considering these two different models we see two alternative needs:

• Codification and automation of repeatable processes to gain cost and efficiency benefits;
• Leverage of the talent and expertise of people to create differentiation.

These two dimensions represent very different kinds of problem areas; as automation increasingly erodes our ability to differentiate through cost and efficiency so the ability to best leverage the talent of our people – which is impossible to codify and replicate – becomes paramount.  As a result we need to concentrate on approaches to service implementation that use technology to automate codified processes whilst delivering collaborative and supportive tooling to enable creative problem solving by people in place of drudgery.  In short we need to have the conceptual notion of services as a unifying set of abstractions around which to build loosely coupled systems, backed up by SOA platforms on which to build automated service realisations and the human-centric view promoted by Web 2.0 to implement those services that are realised by people.

The New People-Oriented Workflow

As a result of these drivers I believe that we will rapidly see much greater emphasis on improving the way that we carry out complex, uncertain tasks which only people can perform.

Whilst transactional tasks will require technology to receive and automatically process the information in well-defined ways, I feel that the main role of technology in human-based services will increasingly be to ‘get out of the way’ and enable people to work creatively to meet their obligations rather than direct them through a fixed set of tasks. I see documents (which is all the contents of a SOAP message or suchlike are) increasingly being surfaced to people in ways that enable them to work directly with the data in a meaningful way; this could be as simple as displaying it using forms technology (rendered at the point of contact using metadata external to the document) or as complex as placing it into a collaboration platform (social software like wikis for example) that enable groups of people to work together. In either case we need tools that enable people to exercise discretion and creativity in responding to service requests whilst supporting collaboration and policy adherence.  This will allow people to work freely on complex problems that require judgement, experience and discretion without the kinds of constraints placed on them by traditional workflow tools and by extension enable a more adaptable, fit for purpose enterprise.

When we look at the majority of current ‘workflow’ tools that dominate the market, however, we see that they are anathema to this model – tightly proceduralised transfer of information through a codified workflow assumes exactly the sort of tasks that should be automated and it doesn’t readily support open, collaborative work between creative individuals; rather it forces them to perform ‘work’ or ‘tasks’ within a set procedure. Such procedures are often overly rigid, quickly overtaken by events and therefore mostly moribund; most importantly, however, they remove the element of human judgement and creativity from the enterprise – increasingly our major source of differentiation.

As a result of these tensions I see Web 2.0 technologies increasingly permeating the enterprise to deliver more lightweight, human-centric support to service realisation, with wikis, presence information, tagging and blogs coming to the fore in an enterprise process and knowledge management context in place of current expensive and ineffectual workflow and knowledge management products.  Whilst SOA technologies will be used to implement both automated services and the macro processes that bring services together, we will use Web 2.0 to deliver a more flexible way of surfacing work to human beings that allows them to meet their obligations in a way that is supportive, easy to use and collaborative.

In addition to these emerging uses within the enterprise such collaborative spaces will become ever more important as customers and partners become codevelopers of our products and services.  The old four walls of the enterprise are rapidly breaking down in the face of decreasing collaboration costs, increasing specialisation and the benefits of what John Hagel and John Seely Brown call ‘productive friction’.  Web 2.0 collaboration technologies can therefore also amplify the benefits of the human talent we have available by linking it to other talent across the whole value chain, leading to more rapid improvement and innovation through diverse experiences and perspectives.

The Death of Process?

On googling around on this subject whilst writing this post I came across a conversation a year ago where Ross Mayfield hypothesised that the types of collaboration that I’m talking about could destroy the whole basis for ‘business process’ by allowing people to rapidly link to the expertise and information necessary to flexibly respond to the demands placed upon them.  This generated a whole slew of comments both for and against and to be clear I just want to state that I do not believe this to be the case.

I feel that far from being less relevant business processes are becoming ever more visible and relevant in the highly connected world that we’re creating – as we design enterprises that leverage business capabilities both within and without the boundaries of the traditional four walls, so we need systematic ways of controlling this coordination to achieve known outputs.  Rather than removing the need for process I think that collaboration technology merely gives us an alternative way of enabling the better leverage of human talent within this systematic framework; at the end of the day we still want to maximise the levels of automation that we can bring to the codifiable elements of our capabilities.  As a result I think we need to look at three main elements in order to understand the connection between these things: 

• Macro processes that describe how capabilities are coordinated in order to realise a value chain;
• Codifiable micro processes that may be used to implement the automatable capabilities; and
• Open, collaborative service realisation technologies – as previously discussed – that enable people to receive service requests in a way which maximises their flexibility and discretion.

Even in this latter category we are not wholly free of the need to adhere to organisational policies – whilst we want to give people the freedom to tackle problems in the best way possible, we also still need to ensure that they do so in a way that is consistent with the policies of the organisation.  Whilst many of these policies can be codified within the service contract that describes the outputs, there will likely also be a need for post-condition type policy evaluation, something discussed frequently by Keith Harrison-Broninski in his work on Human Interaction Management, for an example see here.

As a result I believe that rather than lead to the death of process, collaborative technologies will enhance our ability to deliver flexible processes that fully leverage our available talent by placing human work within the context of a wider value chain in a way that is supportive rather than directive.

Summary

In this post I’ve discussed some of my thoughts around the need to support innovation within human processes by leveraging flexible, collaborative and lightweight tools in place of rigid task-based, automated workflows imposed from the centre.  Such an approach recognises the increasing need for adaptability at the edge and the increasing importance of amplifying human talent as a source of competitive differentiation.  I have also stressed that such tools are a component part of a wider services architecture, however, that has been systematically designed in order to deliver loose organisational coupling; Web 2.0 tools are not a replacement for processes but rather a means of increasing the flexibility of those capabilities within a systematic process that are delivered by people.  In either case my feeling is that we need to move away from an emphasis on tight control and towards a trust and output based model of governance that uses SOA and Web 2.0 to maximise the use of our available talent by allowing people to collaboratively do what needs to be done at the point of need.  Like I said, an end to drudgery.

:: :: :: :: :: :: :: :: :: :: ::

Elements of the Future Business Ecosystem (Part I)

The Wild Outside

It’s no secret that the global business environment has changed radically over the last twenty years due to the effects of globalisation, radically increased competition, pervasive regulation and rapid commoditisation. In addition the increasing information content of products and services is leading to a rapid decline in cycle times as their creation, maintenance and suspension is far more rapid than that of physical goods.  We’re all therefore continually striving to find more innovative propositions with improved service at lower cost - as a result we’ve never needed to be more responsive. Unfortunately, however, most organisations are hampered by organisational, process and technology practices belonging to a less dynamic age, making them unable to adapt within the time and cost parameters increasingly demanded.  These practices make it difficult for us to recognise opportunities in the mass of data with which we’re increasingly being overwhelmed and then frustrate us in any subsequent attempt to adapt, with change being complex, time consuming and hugely expensive.

Whilst these issues have been bubbling away with increasing ferocity under the apparently calm surface of many organisations over the last decade, those people who are continually running ever faster just to keep a lid on things are in increasing trouble – the global environment is set to accelerate further over the next few years.  Technology is increasingly enabling the creation of new business models in support of customers and partners who expect continual service on their own terms – where, when and how they want it – and the speed of change enabled by technology innovation on the Internet represents an almost continuous source of change.

In order to succeed in future, we’re going need to find ways of facing these new realities.  In particular we’re going to have to address the following major issues:

• The need for greater adaptability: The increasing pace of change will force us to adopt strategies that specifically stress adaptability.  Current – mostly dysfunctional – command and control structures that attempt to predict demand and ‘push’ resources will increasingly break down due to the unpredictable nature of the environment.  Organisations will thus need to reinvent themselves as a set of modular business capabilities that provide specific, known and costed services that can be reconfigured on a ‘pull’ basis in response to changing demands; and
• Increasing specialisation: New collaborative technologies and interoperability standards will continue to reduce the transaction costs of collaboration with other organisations.  These changes will enable organisations to replace mediocre internal capabilities with world class services provided by partners, helping us to concentrate on what’s truly important and combat increasing attention scarcity.  We will therefore see increasing specialisation, with organisations who do not outsource non-core concerns finding themselves at an increasing cost and capability disadvantage.

Let’s have a quick look at each of these issues.

The Need For Greater Adaptability

In order to address the need to survive in an increasingly uncertain world organisations need to adopt strategies that specifically stress adaptability – at the end of the day the only way to deal successfully with uncertaintly is to be able to adapt successfully to the changes that occur.  In order to achieve adaptability, however, we must first understand what capabilities are needed to deliver to our stakeholders before considering exactly how all of these component parts combine to deliver value.  Gaining this higher level view of the organisation requires a systematic approach to enterprise design which emphasises the creation and coordination of shared business capabilities in place of hierarchical silos with cross cutting concerns.  Such an approach concentrates on reconceiving the enterprise as a set of collaborating business capabilities which have well understood commitments to the rest of the enterprise and which are firmly placed within the context of the overall value chain.  In particular each capability is expressed only in terms of its commitments – there is no external view of the way in which the capability delivers on these commitments in order to ensure loose organisational coupling.  These changes deliver a set of abstractions that allow senior management to define what the organisation should do without surfacing the mass of how, delivering greater adaptability at both levels.  This brings benefits at a number of levels:

• Organisational leaders are able to concentrate on what they are trying to achieve at the enterprise level by ensuring that they have the correct capabilities and that these capabilities are offering the cost and service levels that the organisation requires.  In addition such leaders are able to concentrate on macro level changes, reconfiguring value chains and renegotiating commitments with the capability owners in response to changes in external demands; 
• Capability owners – including partners – have clear responsibilities and are empowered to innovate within the bounds set by these commitments.  In effect the design of the organisation only constrains the capability owner in terms of required outputs – they are free to produce these outputs in any way they choose as long as they continue to offer competitive services; and
• Organizationally we have much greater adaptability, since change can be understood and managed at multiple levels of abstraction – capability owners can implement changes – for example service improvement or regulatory requirements - without impacting the wider organisation whilst enterprise management have a system of abstractions that allows them to understand the impact of change and to  rework the organisation appropriately.

Although these concepts all sound pretty cool, until recently we’ve lacked a consistent set of abstractions with which to define capabilities, the services they offer and the costs and service levels that they conform to.  In the last few years, however, the concepts of service orientation have emerged. Service orientation allows us to view complex systems – such as an enterprise – as a group of collaborating services that can be coordinated to deliver value, each with its own purpose, contract and service agreement.  These techniques can thus be used – with some extension – to form the basis for the systematic design of the enterprise by helping us to understand and define the business capabilities that are needed whilst deferring their implementation to capability owners. Each business capability can then be delivered using an arbitrary combination of people, physical resources and technology.  This removes the artificial boundaries between different kinds of resources by concentrating on how they combine to deliver service commitments rather than splitting them into – for example – ‘business’ and ‘IT’ and thereby losing the necessary value context.  From an IT perspective this cuts across the traditional view of ‘Enterprise IT’ since capability owners are free to procure any necessary services – potentially including IT – from anyone. 

Increasing Specialisation

Despite the efficiency benefits of specialisation most firms continue to be generalists, executing non-core capabilities in-house rather than rely on other specialists to execute such services on their behalf.  The main reasons for this have been twofold:

• Current organisational structures do not allow for easy outsourcing due to the lack of clarity around discrete business capabilities – it has generally been difficult to untangle consistent threads from the hairball of organisational structure; and
• The transaction costs of collaboration have largely been a barrier to the leverage of external capabilities since such costs have traditionally been sufficiently high as to negate the benefits gained through specialisation.

The former issue will increasingly be resolved through the use of methodologies – such as service orientation – that enable business capability mapping in the search for greater agility.  Transaction costs, however, are a different issue. Traditional economic theory suggests that firms exist to maximise efficiency; in order to do this they organise services internally to themselves in order to minimise the costs of transacting business with third parties. These costs consist of the effort required to find, establish, execute, manage and conclude partnerships for the provision of services.

Until recently these issues have been a sufficient barrier to the emergence of a specialised service provider market, with external transaction costs remaining sufficiently high to prevent large scale outsourcing in all but a few areas of broad applicability (e.g. HR or Payroll).  The increasing capabilities of the Internet are set to change this pattern, however, with key standards facilitating the exchange of information between organisations in a much simpler way.  In particular web service standards – both WS-* and Web 2.0 based - allow documents to be exchanged with partners over the web, forming a basis for a contractual commitment and enabling them to execute some service on our behalf.  When we also add in the more human-collaborative aspects of Web 2.0 and the increasing convergence of communications technology we can see that we have a powerful platform for cross-organisational collaboration. 

By lowering the costs of information exchange and collaboration these developments are opening the door to a change in the nature of the enterprise.  We are headed to a market where organisations will be increasingly virtual - the enterprise will no longer exist to maximise efficiency by minimising transaction costs but to pull together a network of highly efficient, specialised providers in order to deliver an overall proposition.  These changes will be complemented by the modularisation of enterprises, since the definition of a business as a set of services enables us to make clearer decisions about how we realise them.

This process will deliver advantage to organisations in three major ways:

• Enabling tighter focus on the provision of differentiating services: Currently organisational attention spans a myriad of business capabilities, most of which – whilst critical to the overall delivery of value – are not key areas of focus.  Such non-core capabilities represent a diffusion of attention, increasing the information to be absorbed in order  to remain competitive and wasting organisational energy.  As we’re increasingly overwhelmed by information about change – whether competitive, technological or sociological -  attention will become a highly valuable asset and will need to be focused in the correct areas if we are to have any chance of making a sustained impact in our chosen field;
• Boosting wider performance by replacing mediocre internal capabilities with world class services provided by specialised providers: Non-differentiating services will already suffer from a lack of focus and therefore be less effective than those provided by people whose sole concern is to deliver sustained excellence in those capabilities.  As increasing competition and attention scarcity begin to bite we will have even less spare energy within the organisation to devote to these supporting capabilities, leading to rapid degradation.   Those organisations that take the opportunity to leverage 3rd party providers, however, will be exponentially better than those that do not - they will benefit from the ability to focus on their own areas of specialisation whilst simultaneously benefiting from the continual improvement of their specialised partners;  and
• Maximising learning opportunities by working with other specialised providers to improve the overall value chain:  The emergence of extended value chains will see many specialised providers coming together in order to deliver overall propositions to the market.  Where these providers come together there will be abundant opportunities to leverage their different perspectives to generate innovation around the way in which the overall value-chain functions – such innovation will be continuous, sustained and generate improvements at a rate that is inconceivable in today’s static organisations. In addition, the fact that each provider works with many other organisations maximises their opportunities for learning, a stark difference to current internally focused capabilities whose horizons are highly limited.  

Standing back we can see that the move towards specialised service provision further destroys the notion of enterprise IT; as capabilities are unbundled to other providers the only ‘architecture’ that remains critical is adherence to interoperability standards.  Each specialised provider needs to be able to interact with the cloud but can no longer expect to force a unified IT architecture across everyone in the value chain.

Conclusions

In this first post I have considered some of the broad changes that I think are going to have a major impact on enterprises but I’ve really only brushed some of the implications from an IT perspective.  I guess the big take away from this initial post is that organisations – and therefore enterprise IT – are entering a period of fragmentation as unbundling occurs.  In future posts I’ll look at the potential effects of such fragmentation before going on to talk about the resulting future landscape from a business and IT perspective.

:: :: :: :: :: :: :: :: :: ::