SOA, Web 2.0 and the End of Drudgery

20 Jun
The Growth of Services

One of the great boons of the growth of XML has been the way in which it has transformed integration – increasingly we’re seeing information flowing around the Internet as a set of self-describing documents that contain their own context.  Many of the services that process these documents may be automated – like many SOA or Web 2.0 services that we see today – but many will actually be routes to people (like Amazon’s Mechanical Turk – genius).  In this post I wanted to just explore some of the implications of this, since I believe that human-provided services will come to have very different characteristics from those provided by machines. 

The Growth of Drudgery

Over the last twenty years we have seen increasing concentration on the automation of transactional tasks (i.e. things which can be ‘proceduralised’ and repeated).  This has yielded enormous cost and efficiency benefits but we’re increasingly finding ourselves passing those benefits onto consumers.  As a result cost-cutting and efficiency are becoming a way to remain in the game rather than a source of competitive advantage.  In fact many people – most famously Nick Carr – argue that the use of IT to automate rote processes no longer yields any opportunity for differentiation, since all companies have access to the same technical capabilities and the work being automated is itself highly proceduralised and therefore by it’s nature easy to replicate.  Part of the rush to automate and proceduralise, however, has seen organisations place considerable emphasis on ‘workflow’ technologies that attempt to box people into specific tasks within an overall process, with little chance for innovation or creativity.  Such tools generally emphasise control and repeatability over contextual decision making – in a Taylorist mould – and therefore limit the freedom of the people and teams receiving work requests.  As a result we have seen a huge increase in the amount of work that is fixed, detailed and monitored, resulting in what I would call drudgery. 

Talent as the New Competitive Differentiation

In parallel with the realisation that increasing automation is reducing the scope for differentiation in transactional processes we have started to realise that – unsurprisingly – real, hard to recreate differentiation is found in the knowledge and talent of our people.  Most organisations make little or no attempt to recognise and promote the value of talent, however, and in fact encase people within industrial age processes that attempt to control their every response irrespective of context, reducing the scope for them to creatively tackle problems and find solutions whilst simultaneously reducing the adaptability and initiative of the enterprise as a whole.  Despite the continued belief in many companies that ‘the centre’ knows best, we’re actually seeing a movement towards increasing federation; as the world becomes a more uncertain and dynamic place, so the ability to compose the ‘perfect’ plan from the centre and then just get people to execute it breaks down completely.  As a result we need to move decision making to the edges of process boundaries through the coordination of loosely coupled services.  The benefit of this federated approach is that it moves the decisions on how to meet goals much closer to the people that have the knowledge to best decide what needs to be done whilst maintaining governance through concentration on outputs and KPIs in place of tight control of the tasks completed.  This is a major shift in management philosophy, relying as it does on service contracts, trust and a belief in people in place of absolute control of every stage of a process.  The return, however, is that we get more appropriate, adaptable and innovative services that leverage the wealth of human talent available to us in place of smothering it. 

As a result one of the major tasks facing all industry sectors is to yes, continue to automate rote tasks to drive efficiencies but also to realise that real, difficult to replicate competitive advantage will come from the way in which we allow creative people to use information at the edge in providing services both to customers and back into the organisation.  As transactional jobs are increasingly automated, outsourced and commoditised (since it is difficult to create non-replicateable differentiation in these due to the fact that such processes can be easily codified and replicated) we will need to break down our old notions of control and embrace trust based models to support creative, customer relationship or problem resolution processes where real differentiation will be delivered which is difficult to recreate (due to the fact that it is dependent on talent, contextual awareness and experience – three things which cannot be codified).

The Alternative Growth of Extreme Federation

Whilst much of this work has been going on we’ve seen an alternative approach to command and control workflow systems growing on the Internet.  Increasingly complex applications, businesses and social networking sites have sprung up from sets of loosely coupled people and systems with the emphasis much more firmly placed on expectations and trust.  In such extreme federation people concentrate on the required outcomes whilst wholly devolving the way in which those outcomes are achieved to friends, partners or 3rd parties.  In solving particular problems, people come together within newsgroups, on wikis, via blogs or (even – shock) via email, collaborating to solve problems together within their sphere of expertise.  One of the most extreme examples of such community based activity is the open source software movement, where high levels of trust and collaboration yield levels of innovation, creativity and quality that often outperform traditional command and control methods of management within product development companies.  One of the key aspects of this movement, however, is the leveraging of human capital to take advantage of the well of creativity and talent that this represents instead of merely seeing people as drones to effect some part of a pre-planned process.  Many of these changes have resulted in the emergence of the group of concepts known as Web 2.0, with key elements that support people, their collaborations and their access to services in a personalised way.  By supporting such collaboration these technologies are ushering in an age of accelerated innovation, by bringing together many experiences and perspectives and allowing the people involved to interact, argue and learn, building new value faster than ever before.

Human Vs Machine

In considering these two different models we see two alternative needs:

  • Codification and automation of repeatable processes to gain cost and efficiency benefits;
  • Leverage of the talent and expertise of people to create differentiation.

These two dimensions represent very different kinds of problem areas; as automation increasingly erodes our ability to differentiate through cost and efficiency so the ability to best leverage the talent of our people – which is impossible to codify and replicate – becomes paramount.  As a result we need to concentrate on approaches to service implementation that use technology to automate codified processes whilst delivering collaborative and supportive tooling to enable creative problem solving by people in place of drudgery.  In short we need to have the conceptual notion of services as a unifying set of abstractions around which to build loosely coupled systems, backed up by SOA platforms on which to build automated service realisations and the human-centric view promoted by Web 2.0 to implement those services that are realised by people.

The New People-Oriented Workflow

As a result of these drivers I believe that we will rapidly see much greater emphasis on improving the way that we carry out complex, uncertain tasks which only people can perform.

Whilst transactional tasks will require technology to receive and automatically process the information in well-defined ways, I feel that the main role of technology in human-based services will increasingly be to ‘get out of the way’ and enable people to work creatively to meet their obligations rather than direct them through a fixed set of tasks. I see documents (which is all the contents of a SOAP message or suchlike are) increasingly being surfaced to people in ways that enable them to work directly with the data in a meaningful way; this could be as simple as displaying it using forms technology (rendered at the point of contact using metadata external to the document) or as complex as placing it into a collaboration platform (social software like wikis for example) that enable groups of people to work together. In either case we need tools that enable people to exercise discretion and creativity in responding to service requests whilst supporting collaboration and policy adherence.  This will allow people to work freely on complex problems that require judgement, experience and discretion without the kinds of constraints placed on them by traditional workflow tools and by extension enable a more adaptable, fit for purpose enterprise.

When we look at the majority of current ‘workflow’ tools that dominate the market, however, we see that they are anathema to this model – tightly proceduralised transfer of information through a codified workflow assumes exactly the sort of tasks that should be automated and it doesn’t readily support open, collaborative work between creative individuals; rather it forces them to perform ‘work’ or ‘tasks’ within a set procedure. Such procedures are often overly rigid, quickly overtaken by events and therefore mostly moribund; most importantly, however, they remove the element of human judgement and creativity from the enterprise – increasingly our major source of differentiation.

As a result of these tensions I see Web 2.0 technologies increasingly permeating the enterprise to deliver more lightweight, human-centric support to service realisation, with wikis, presence information, tagging and blogs coming to the fore in an enterprise process and knowledge management context in place of current expensive and ineffectual workflow and knowledge management products.  Whilst SOA technologies will be used to implement both automated services and the macro processes that bring services together, we will use Web 2.0 to deliver a more flexible way of surfacing work to human beings that allows them to meet their obligations in a way that is supportive, easy to use and collaborative.

In addition to these emerging uses within the enterprise such collaborative spaces will become ever more important as customers and partners become codevelopers of our products and services.  The old four walls of the enterprise are rapidly breaking down in the face of decreasing collaboration costs, increasing specialisation and the benefits of what John Hagel and John Seely Brown call ‘productive friction’.  Web 2.0 collaboration technologies can therefore also amplify the benefits of the human talent we have available by linking it to other talent across the whole value chain, leading to more rapid improvement and innovation through diverse experiences and perspectives.

The Death of Process?

On googling around on this subject whilst writing this post I came across a conversation a year ago where Ross Mayfield hypothesised that the types of collaboration that I’m talking about could destroy the whole basis for ‘business process’ by allowing people to rapidly link to the expertise and information necessary to flexibly respond to the demands placed upon them.  This generated a whole slew of comments both for and against and to be clear I just want to state that I do not believe this to be the case.

I feel that far from being less relevant business processes are becoming ever more visible and relevant in the highly connected world that we’re creating – as we design enterprises that leverage business capabilities both within and without the boundaries of the traditional four walls, so we need systematic ways of controlling this coordination to achieve known outputs.  Rather than removing the need for process I think that collaboration technology merely gives us an alternative way of enabling the better leverage of human talent within this systematic framework; at the end of the day we still want to maximise the levels of automation that we can bring to the codifiable elements of our capabilities.  As a result I think we need to look at three main elements in order to understand the connection between these things: 

  • Macro processes that describe how capabilities are coordinated in order to realise a value chain;
  • Codifiable micro processes that may be used to implement the automatable capabilities; and
  • Open, collaborative service realisation technologies – as previously discussed – that enable people to receive service requests in a way which maximises their flexibility and discretion.

Even in this latter category we are not wholly free of the need to adhere to organisational policies – whilst we want to give people the freedom to tackle problems in the best way possible, we also still need to ensure that they do so in a way that is consistent with the policies of the organisation.  Whilst many of these policies can be codified within the service contract that describes the outputs, there will likely also be a need for post-condition type policy evaluation, something discussed frequently by Keith Harrison-Broninski in his work on Human Interaction Management, for an example see here.

As a result I believe that rather than lead to the death of process, collaborative technologies will enhance our ability to deliver flexible processes that fully leverage our available talent by placing human work within the context of a wider value chain in a way that is supportive rather than directive.

Summary

In this post I’ve discussed some of my thoughts around the need to support innovation within human processes by leveraging flexible, collaborative and lightweight tools in place of rigid task-based, automated workflows imposed from the centre.  Such an approach recognises the increasing need for adaptability at the edge and the increasing importance of amplifying human talent as a source of competitive differentiation.  I have also stressed that such tools are a component part of a wider services architecture, however, that has been systematically designed in order to deliver loose organisational coupling; Web 2.0 tools are not a replacement for processes but rather a means of increasing the flexibility of those capabilities within a systematic process that are delivered by people.  In either case my feeling is that we need to move away from an emphasis on tight control and towards a trust and output based model of governance that uses SOA and Web 2.0 to maximise the use of our available talent by allowing people to collaboratively do what needs to be done at the point of need.  Like I said, an end to drudgery.

Advertisements

3 Responses to “SOA, Web 2.0 and the End of Drudgery”

Trackbacks/Pingbacks

  1. Is BPEL4People really for The People by The People? « IT Blagger 3.0 - July 12, 2007

    […] is the desirability of the kind of process support that BPEL can implement for people.  As I discussed in a previous post, increasing reliance on talent and the collaboration of talent for competitive differentiation […]

  2. Stability, Relationships and Institutional Innovation as Pillars of Success « IT Blagger 3.0 - February 5, 2008

    […] distributed co-creation and innovation.  It may be that some of the tools are already here – I’ve written about using Web 2.0 techniques to leverage talent, for example – but our understanding of the practices and processes that will enable us to really exploit these […]

  3. Is Social Media Rubbish? « IT Blagger 3.0 - July 8, 2011

    […] “opening up” a business and allowing it to participate in wider value creation networks.  As I’ve discussed before, however, tooling will only exist to support talented people undertaking creative processes within […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: