Why Amazon Dedicated Instances Is No Big Deal… and Why it Really Is

Why It’s No Big Deal

I was interested yesterday in the amount of excitement that Amazon’s announcement of dedicated instances caused.  To me this seems like a sensible move from a public cloud provider in order to counter the widespread belief in large enterprises that they need to be physically as well as logically separate.  This represents a maturation of public cloud offerings in much the way I’ve discussed in the past and demonstrates that public clouds can evolve to provide the kinds of additional security enterprises (currently) perceive that they require.  This can only be a good thing as bit by bit public cloud companies like Amazon are removing the FUD generated by IT departments and traditional IT service companies and commoditising this completely unimportant stuff so that we can all move on and talk about the real business opportunities of the cloud.

Beyond the satisfaction of seeing another ‘roadblock’ item ticked off the list, however, in technical terms this seems like no big deal to me.  Essentially Amazon are offering you the ability to have a compute instance that takes up all of the resources on a single physical machine (whether you need all of those resources or not) and so fits into their existing framework of differently sized compute instances.  From that perspective it doesn’t feel groundbreaking as it merely ‘tweaks’ their existing model to mark a physical machine as ‘full’ (I’m obviously over-simplifying but intentionally so).

For these reasons I don’t subscribe to the idea that this ‘breaks’ the cloud model or is in some way not multi-tenant since there are no upfront costs for hardware, you still pay for what you use and the whole platform infrastructure is still shared.  The only difference is that you can choose to have some of your compute instances marked as requiring the resources of a complete physical machine.  Interestingly this is also my understanding of how Azure works – their compute instances are broken down into subsets of a physical machine; if you take the biggest instance you’ve essentially got that machine to yourself (although I guess that’s a side-effect of design rather than a conscious offering as per Amazon).

Why It Really Is a Big Deal

So technically we can consider this move to be a relatively small deal even though it is perceived by many as a potentially game changing additional capability.

And frankly that’s the big deal.

Most ‘private cloud’ initiatives from enterprise IT or traditional IT service vendors start from the perspective of trying to protect existing models by merely adding virtualisation and management to existing estates.  They are trying to extend an old model of enterprise IT in the vague hope that it will give them the same benefits as public cloud.  The two things are not vaguely equivalent, however, and they are hugely underestimating the differences.  There is no way that such efforts will result in something as sophisticated as Amazon’s public cloud, something that has been built from the bottom up as an optimised, integrated and low cost service that commoditises many complex products, processes and infrastructure into a single platform that caters for general usage.  There’s just too much distraction and baggage (systems wise and business model wise) for such efforts to ever succeed.  It’s not even putting lipstick on a pig but more like fluffing its muddy hair a little and half closing your eyes.

On the other hand Amazon have proven that not only are they able to build a platform that can operate at high scale and low cost for a non-enterprise market but that this new model can also be extended to cater for enterprise needs.  And they can do this competitively because they have done the spade work required to serve a lower cost market.  Adding some features to separate tenants using your ability to manage a commodity platform (for example) is much easier than trying to work out how to strip huge costs from traditional models of ownership.  This is the traditional pattern of disruptive innovation where a competitor seen as unfit for purpose by demanding users builds solutions that are far more capable and cost effective for the low end before leveraging these benefits upwards to oust incumbent suppliers at the upper end of the market.

In evolutionary terms cloud is a point of ‘punctuated equilibrium’ where the criteria for fitness to survive changes rapidly – whereas previously an ability to afford the ownership of complex, bespoke IT was a competitive advantage, it has now become a distinct disadvantage for everything except a small set of differentiating processes that represent core capabilities.  Furthermore in such a rapidly changing environment companies like Amazon who are focused around a key part of the emerging value web (i.e. an infrastructural business model focused on a commoditised IT platform) can rapidly evolve based on the selection criteria of the market, leaving traditional participants trapped and encumbered by outmoded business models.

Today this traditional end of the market is populated by enterprise IT departments, software providers, hardware providers and IT service providers – all of whom will increasingly see huge losses of business when judged for fitness against commoditised public cloud platforms.  Essentially many such participants will literally have no competitive offering as they have been prevented from making the shift by their own evolution of traits specialised for a previous environment.  As a result expect to see even more rabid ‘cloud washing’ and pushing of ‘private clouds’ by such vendors as the literal need for them ebbs away – effectively many of them will need to extend existing business for as long as possible while bringing new cloud platforms to market or deciding to cede this business completely.

Effectively companies who to date have been incapable of changing are being eaten from underneath and must decide whether they try to compete (in which case they need significant business, structural and asset realignment) or retreat into other business types that don’t compete with platforms (so consulting or implementation of vertical business services for instance – for IT departments see here for a suggestion).

Why It’s a Really Big Deal For You

For me this announcement is further proof of my long standing belief that you cannot build a successful cloud platform as an extension of old models and from within a business for whom it is not their core concern.  Rather successful providers must be ruthlessly focused on building a new cloud platform that integrates, optimises and simplifies complex technologies into a low cost service.  As a cloud platform consumer you should therefore think very carefully about the implications of this and consider whether buying that hypervisor software, hardware and services for a private cloud implementation will really get you an Amazon of your very own – or just further baggage for your business to drag along.

UPDATE:  Added link to Amazon announcement at the beginning of the post as… I forgot.

What’s the Future of SOA?

EbizQ asked last week for views on the improvements people believe are required to make SOA a greater success.  I think that if we step back we can see some hope – in fact increasing necessity – for SOA and the cloud is going to be the major factor in this.

If we think about the history of SOA to date it was easy to talk about the need for better integration across the organisation, clearer views of what was going on or the abstract notion of agility. Making it concrete and urgent was more of an issue, however. Whilst we can discuss the ‘failure’ of SOA by pointing to a lack of any application of service principles at a business level (i.e. organisationally through some kind of EA) this is really only a symptom and not the underlying cause. In reality the cause of SOA failure to date has been business inertia – organisations were already set up to do what they did, they did it well enough in a push economy and the (understandable) incentives for wholesale consideration of the way the business worked were few.

The cloud changes all of this, however. The increasing availability of cloud computing platforms and services acts as a key accelerator to specialisation and pull business models since it allows new entrants to join the market quickly, cheaply and scalably and to be more specialised than ever before. As a result many organisational capabilities that were economically unviable as market offerings are now becoming increasingly viable because of the global nature of cloud services. All of these new service providers need to make their capabilities easy to consume, however, and as a result are making good use of what people are now calling ‘apis’ in a web 2.0 context but which are really just services; this is important as one of the direct consequences of specialisation is the need to be hooked into the maximum number of appropriate value web participants as easily as possible.

On the demand side, as more and more external options become available in the marketplace that offer the potential to replace those capabilities that enterprises have traditionally executed in house, so leaders will start to rethink the purpose of their organisations and leverage the capabilites of external service providers in place of their own.

As a result cloud and SOA are indivisable if we are to realise the potential of either; cloud enables a much broader and more specialised set of business service providers to enter a global market with cost and capability profiles far better than those which an enterprise can deliver internally. Equally importantly, however, they will be implicitly (but concretely) creating a ‘business SOA catalogue’ within the marketplace, removing the need for organisations to undertake a difficult internal slog to re-implement or re-configure outdated capabilities for reuse in service models. Organisations need to use this insight now to trigger the use of business architecture techniques to understand their future selves as service-based organisations – both by using external services as archtypes to help them understand the ways in which they need to change and offer their own specialised services but also to work with potential partners to co-develop and then disaggregate those services in which they don’t wish to specialise in future.

Having said all that to set the scene for my answer(!) I believe that SOA research needs to be focused on raising the concepts of IT mediated service provision to a business level – including concrete modelling of business capabilities and value webs – along with complex service levels, contracts, pricing and composition – new cloud development platforms, tooling and management approaches linked more explicitly to business outcomes – and which give specialised support to different kinds of work – and the emergence of new 3rd parties who will mediate, monitor and monetise such relationships on behalf of participants in order to provide the required trust.

All in all I guess there’s still plenty to do.

Is Your Business Right For The Cloud?

I left a short comment on the ebizq website a couple of days ago in response to the question ‘is the cloud right for my business?’

I thought I’d also post an extended response here as I strongly believe that this is the wrong question.  Basically I see questions like this all the time and they are always framed and answered at the wrong level, generating a lot of heat – as people argue about the merits of public vs private infrastructures etc – but little insight.  Essentially there are a number of technology offerings available which may or may not meet the specific IT requirements of a business at a particular point in time.  Framed in the context of traditional business and IT models the issues raised often focus on the potentially limited benefits of a one to one replacement of internal with external capability in the context of a static business.  Its usually just presented as a question of whether I provide equivalent IT from somewhere else (usually somewhere dark and scary) or continue to run it in house (warm, cuddly and with tea and biscuits thrown in).  The business is always represented as static and unaffected by the cloud other than in the degree to which its supporting IT is (marginally) better or (significantly) worse.

If the cloud was truly just about taking a traditional IT managed service (with some marginal cost benefit) vs running it in house – as is usually positioned – then I wouldn’t see the point either and would remain in front of the heater in my carpet slippers with everyone else in IT.  Unfortunately for people stuck in this way of thinking  – and the businesses that employ them – the cloud is a much, much bigger deal.

Essentially people are thinking too narrowly in terms of what the cloud represents.  It’s not about having IT infrastructure somewhere else or sourcing ‘commodity applications’ differently.  These may be the low hanging fruit visible to IT folks currently but they are a symptom of the impact of cloud and not the whole story.

The cloud is all about the falling transaction costs of collaboration and the current impact on IT business models is really just a continuation of the disruptions of the broader Internet.  As a result whilst we’re currently seeing this disruption playing out in the IT industry (through the commoditisation of technology and a move towards shared computing of all kinds) it is inevitable that other industry disruptions will follow as the costs of consuming services from world-class partners plummets and the enabling technology becomes cheaper, more configurable, more social and more scalable as a result of the reformation of the IT industry.

Essentially all businesses need to become more adaptive, more connected and more specialised to succeed in the next ten years and the cloud will both force this and support it.  Getting your business to understand and plan for these opportunities – and having a strong cloud strategy to support them – is probably the single most important thing a CIO can do at the moment.  Not building your own ‘private cloud’ with no expertise or prior practice to package or concentrating on trying to stop business colleagues with an inkling of the truth from sourcing cloud services more appropriate to their needs.  Making best use of new IT delivery models to deliver truly competitive and world-class business capabilities for the emerging market is the single biggest strategic issue facing CIOs and the long term health of the businesses they serve.  There is both huge untapped value and terrific waste languishing inside existing business structures and both can be tackled head on with the help of the cloud.  Optimising the limited number of business capabilities that remain in a business’s direct control – as opposed to those increasingly consumed from partners – will be a key part of making reformed organisations fit for the new business ecosystem.

As a result the question isn’t whether the cloud is or will be ‘right’ for your business but rather how ‘right’ your business will be for the cloud. Those organisations that fail to take a broader view and move their business and technical models to be ‘right’ for the cloud will face a tough struggle to survive in a marketplace that has evolved far beyond their capabilities.

iPad not harbinger of PC doom according to Steve Jobs

After having put some time into thinking about people’s discomfort with the iPad a couple of weeks ago I was interested in this brief article in AppleInsider where Steve Jobs admits that the notion of a post-PC era is ‘uncomfortable’ for many people – a subject that I touched on in my post.  Jobs’ comments appear to support my own impressions that this is really just a maturation of the industry, a democratisation of access to computing for the masses and that it won’t undermine traditional computing for those with the necessary skills.  This should be a relief to people who worry that such devices will replace computers and thereby destroy the ability of individuals to be technically “generative”.   I also basically agree with his summary of tablets as a new form factor that replaces the need for a PC for many people, that PCs will continue to exist and that more choice is good (and although I still don’t agree with the Apple business model – and feel that it will suffer as other people replicate their innovations in more open ecosystems – only one of us is obscenely rich ).

More broadly my gut feel is that as the interfaces and capabilities of tablets increase in sophistication so we will be able to encourage more ‘vertical’ and ‘individual’ creativity and “generativity” in the population as a whole.  These people won’t be using the same tools as those we’ve had to learn to create through PC use but then they also won’t need that lower level, general-purpose control over raw computing that many people have had to learn merely to pursue higher level interests.  There will still be plenty of IT work – in fact more than ever – implementing applications and services to help these newly liberated consumers ignore the underlying computer and be creative within their own domains.

Cloud Platforms and Future Middleware

I’m going to try and break the habit of a lifetime in this ‘second life’ of my blog and post the odd ‘peppy’ comment on things I’ve seen as well as getting sucked into long analyses :-p

In that spirit I thought I’d just comment on a post I saw today by John Rymer at Forrester; essentially John was expressing some mild disappointment at a discussion about future app servers he was involved in and suggesting that the future of these products needs to be radically different in a connected, cloud environment.  I completely agreed with his points about more lightweight, specialised and virtualised ‘containers’ and this reflected the work I discussed in one of my older posts, where I talked about the need to use virtual templates, lightweight product and framework configurations, specific patterns and metadata plus domain specific languages and factories in pursuit of IT industrialisation.  Such lightweight and specialised containers for service realisation help to make developers more productive but also enable much greater agility and efficiency in resource usage by allowing each such service to change and scale according to its purpose and needs independent of the others.  In this sense I understand the feeling of one person who left a comment who described such platforms in terms of a fabric; this is probably an apt description given that you will have independent, specialised services bound to specific lightweight containers, ‘floating’ on a virtual infrastructure and collaborating with others to realise wider intent.  At heart a lot of John’s post was about simplifying, downsizing and specialising containers for different kinds of services and so I heartily agreed with his sentiments on the matter.

Business Enablement as a Key Cloud Element

After finally posting my last update about ‘Industrialised Service Delivery’ yesterday I have been happily catching up with the intervening output of some of my favourite bloggers.

One post that caught my eye was a reference from Phil Wainwright – whilst he was talking about the VMForce announcement – to a post he had written earlier in the year about Microsoft’s partnership with Intuit.  Essentially one of his central statements was related directly to the series of posts I completed yesterday (so part 1, part 2 and part 3):

“the breadth of infrastructure <required for SaaS> extends beyond the development functionality to embrace the entirely new element of service delivery capabilities. This is a platform’s support for all the components that go with the as-a-service business model, including provisioning, pay-as-you-go pricing and billing, service level monitoring and so on. Conventional software platforms have no conception of these types of capability but they’re absolutely fundamental to delivering cloud services and SaaS applications”.

This is one of the key points that I think is still – inexplicably – lost on many people (particularly people who believe that cloud computing is primarily about providing infrastructure as a service).  In reality the whole world is moving to service models because they are simpler to consume, deliver clearer value for more transparent costs and can be shared across organisations to generate economies of scale.  In fact ‘as a service’ models are increasingly not going to be an IT phenomenon but also going to extend to the way in which businesses deal with each other across organisational boundaries.  For the sale and consumption of such services to work, however, we need to be able to ‘deliver’ them; in this context we need to be able to market them, make them easy to subscribe to, manage billing and service levels transparently for both the supplier and consumer and enable rapid change and development over time to meet the evolving needs of service consumers.  As a result anyone who wants to deliver business capabilities in the future – whether these are applications or business process utilities – will need to be able to ensure that their offering exhibits all of these characteristics. 

Interestingly these ‘business enablement’ functions are pretty generic across all kinds of software and services since they essentially cover account management, subscription, business model definition, rating and billing, security, marketplaces etc etc (i.e. all of the capabilities that I defined as being required in a ‘Service Delivery Platform’).  In this context the use of the term ‘Service Delivery Platform’ in place of cloud or PaaS was deliberate; what next generation infrastructures need to do is enable people to deliver business services as quickly and as robustly as possible, with the platforms themselves also helping to ensure trust by brokering between the interests of consumers and suppliers through transparent billing and service management mechanisms.

This belief in service delivery is one of the reasons I believe that the notion of ‘private clouds’ is an oxymoron – I found this hoary subject raised again on a Joe McKendrick post after a discussion on ebizQ – even without the central point about the obvious loss of economies of scale; essentially  the requirement to provide a whole business enablement fabric to facilitate cross organisational service ecosystems – initially for SaaS but increasingly for organisational collaboration and specialisation – is just one of the reasons I believe that ‘Private Clouds’ are really just evolutions of on-premise architecture patterns – with all of the costs and complexity retained – and thus purely marketecture.  When decreasing transaction costs are enabling much greater cross organisational value chains the benefits of a public service delivery platform are immense, enabling organisations to both scale and evolve their operations more easily whilst also providing all of the business support they need to offer and consume business services in extended value chains.  Whilst some people may think that this is a pretty future-oriented reason to not like the notion of private clouds, for completeness I will also say that to me  – in the sense of customer owned infrastructures – they are an anachronism; again this is just an extension of existing models (for good or ill) and nothing to do with ‘cloud’.  It is only the fact that most protagonists of such models are vendors with very low level maturity offerings like packaged infrastructure and/or middleware solutions that makes it viable, since the complexity of delivering true private SDP offerings would be too great (not to mention ridiculously wasteful).  In my view ‘private clouds’ in the sense of end organisation deployment is just building a new internal infrastructure (whether self managed or via a service company) sort of like the one you already already have but with a whole bunch of expensive new hardware and software (so 90% of the expense but only 10% of the benefits). 

To temper this stance I do believe that there is a more subtle, viable version of ‘privacy’ that will be supported by ‘real’ service delivery platforms over time – that of having a logically private area of a public SDP to support an organisational context (so a cohesive collection of branded services, information and partner integrations – or what I’ve always called ‘virtual private platforms’).  This differs greatly from the ‘literally’ private clouds that many organisations are positioning as a mechanism to extend the life of traditional hardware, middleware or managed service offerings – the ability of service delivery platforms to rapidly instantiate ‘virtual’ private platforms will be a core competency and give the appearance and benefits of privacy whilst also maintaining the transformational benefits of leveraging the cloud in the first place.  To me literally ‘private clouds’ on an organisations own infrastructure – with all of their capital expense, complexity of operation, high running costs and ongoing drag on agility – only exist in the minds of software and service companies looking to extend out their traditional businesses for as long as possible. 

Industrialised Service Delivery Redux III

It’s a bit weird editing this more or less complete post 18 months later but this is a follow on to my previous posts here and here.  In those posts I discussed the need for much greater agility to cope with an increasingly unpredictable world and ran through the ways in which we can industrialise IT provision to focus on tangible business value and rapid realisation of business capability.  This story relied upon the core notion that technology is no longer a differentiator in and of itself and thus we just need workable patterns that meet our needs for particular classes of problem – which in turn reduces the design space we need to consider and allows increasing use of specialised platforms, templates and development tools.

In this final post I will discuss the notion that such standardisation calls into question the need to own such technology at all; essentially as platforms and tools become more standardised and available over the network so the importance of technology moves to access rather than ownership.

Future Consolidation

One of the interesting things from my perspective is that once you start to build out an asset-based business – like a service delivery platform – it quickly becomes subject to economies of scale.

It is rapidly becoming plain, therefore, that game changing trends such as:

• Increasing middleware consolidation around traditional ‘mega platform’ providers;
• Flexible infrastructure enabled by virtualisation technology;
• Increasingly powerful abstractions such as service-orientation;
• The growing influence of open source software and collaborating communities; and
• The massively increased interconnectivity enabled by the web.

are all going to combine to change not just the shape of the IT industry itself but increasingly all industries; essentially as IT moves to service models so organisations will need to reshape themselves to align with these new realities, both in terms of their use of IT but also in terms of finding their distinctive place within their own disaggregating business ecosystems.

From a technology perspective it is therefore clear that these forces are combinatory and lead to accelerating commoditisation.  The implication of this acceleration is that decreasing differentiation should lead to increased consolidation as organisations no longer need to own and operate their own IT when such IT incurs cost and complexity penalties without delivering differentiation.

In a related way such a shift by organisations to shared IT platforms is also likely to be an amplifying trend; as we see greater platform consolidation – and hence decreasing differentiation to organisations owning their own IT – so will laggard organisations become less competitive as a result of their expensive and high drag IT relative to their low cost, fleet of foot competitors.  Such organisations will then also seek to transition, eventually creating a tipping point at which ownership of IT becomes an anachronism.

From the supply perspective we can also see that as platforms become less differentiating and more commoditised they also become subject to increasing economies of scale – from an overall market perspective, therefore, offering platforms as a service becomes a far more effective use of capital than the creation and ownership of an island of IT, since scale technologies drift naturally towards consolidation.  There are some implications to this for the IT industry given the share of overall IT spend that goes on repeated individual installation and consulting for software and hardware but we shall leave that for another post.

As a result of these trends it is highly likely that we will see platform as a service propositions growing in influence fairly rapidly.  Initially these platforms are likely to be infrastructure-oriented and targeted at new SaaS providers or transitioning ISVs to lower the cost of entry but I believe that they will eventually expand to deliver the full business enablement support required by all organisations that need to exist in extended value webs (i.e. eventually everyone).  These latter platforms will need to have all of the capabilities I discussed in the previous post and will be far beyond the technology-centric platforms envisaged by the majority of emerging platform providers today.  Essentially as everybody becomes a service provider (or BPU in other terms) in their particular business ecosystem so they will need to rapidly realise, commercialise, manage and adapt the services they offer to their value webs.  In this latter scenario I believe that organisations will be caught in the jaws of a vise – the unbundling of capability to SaaS or other BPU providers to allow them to specialise and optimise the overall value stream will see their residual IT costs rocket as there are less capabilities to share it around; at the same time economies of scale produced by IT service companies will see the costs of platform as a service offerings plummet and make the transition a no brainer.

So what would a global SDP look like?

Well remarkably like the one I showed in my previous posts given that I was leading up to this point, lol.  The first difference is that the main bulk of the platform is now explicitly deployed in the cloud – and it’ll obviously need to scale up and down smoothly and at low cost.  In addition all of the patterns that we discussed in my previous post will need to support multi-tenancy and such patterns will need to be built into the tools and factories that we will use to create systems optimised to run on our Service Delivery Platform.

At the same time the service factory becomes a way of enabling the broadest range of stakeholders to rapidly and reliably create services and applications that can be deployed to our platform – in fact it moves from being “just” an interesting set of tools to support industrialised capability realisation to being one of the main battlegrounds for PaaS providers trying to broaden their subscriber base by increasing the fidelity of realisation and reducing the barrier of entry to the lowest level possible.

Together the cloud platform and associated service factory will be the clear option of choice for most organisations, since it will yield the greatest economies of scale to the people using it.

One last element on this diagram that differentiates it from the earlier one is the on-premise ‘customer service platform’. In this context there is still a belief in many quarters that organisations will not want to physically share space and hardware with other people – they may be less mature, they may not trust sufficiently or they may genuinely have reasons why their data and services are so important that they are willing to pay to host them separately.  In the long term I do not subscribe to this view and to me the notion of ‘private clouds’ – outside of perhaps government and military use cases – is oxymoronic and at best a transitional situation as people learn to trust public infrastructures.  On the other hand whilst this may be playing with semantics I can see the case for ‘virtual private clouds’ (i.e. logically ring fenced areas of public clouds) that give the appearance and majority of benefits of being private through ‘soft’ partitioning (i.e. through logical security mechanisms) whilst allowing the retention of economies of scale through avoidance ‘hard’ partitioning (i.e. through separate physical infrastructure).  Indeed I would state that such mechanisms for making platforms appear private (including whitelabelling capabilities) will be necessary to support the branding requirements of resellers, systems integrators and end organisations.  For the sake of completeness, however, I would position transitional ‘private clouds’ as reduced functionality versions of a Service Delivery Platform that simply package up some hardware but leave the majority of the operational and business support – along with things like backup and failover – back at the main data centres of the provider in order to create an acceptable trade-off in cost.

Summary

So in this final post I have touched on some of the wider changes that are an implication of technology commoditisation and the industrialisation of service realisation.  For completeness I’ll recap the main messages from the three posts:

• In post one I discussed how businesses are going to be forced to become much more aware of their business capabilities – and their value – by the increasingly networked and global nature of business ecosystems.  As a result they will be driven to concentrate very hard on realising their differentiating capabilities as quickly, flexibly and cost effectively as possible; in addition they will need to deliver these capabilities with stringent metrics.  This has some serious implications for the IT industry as we will need to shift away from a technology focus (where the client has to discover the value as a hit and miss emergent process) to one where we can demonstrate a much more mature, reliable and outcome based proposition. To do this we’ll need to build the platforms to realise capabilities effectively and in the broadest sense.
• In post two I discussed how industrialisation is the creation and consistent application of known patterns, processes and infrastructures to increase repeatability and reliability. We might sacrifice some flexibility but increasing commoditisation of technology makes this far less important than cost effectiveness and reliability. When industrialising you need to understand your end to end process and then do the nasty bit – bottom up in excruciating detail.
• Finally in post three I have discussed my belief that increasing standardisation of technology will lead to accelerating platform consolidation.  Essentially as technology becomes less differentiating and subject to economies of scale it’s likely that IT ownership and management will be less attractive. I believe, therefore, that we will see increasing and accelerating activity in the global Service Delivery Platform arena and that IT organisations and their customers need to have serious, robust and viable strategies to transition their business models.

Industrialised Service Delivery Redux II

In my previous post I discussed the way in which our increasingly sophisticated use of the Web is creating an unstoppable wave of change in the global business environment.  This resulting acceleration of change and expectation will require unprecedented organisational speed and adaptability whilst simultaneously driving globalisation and consumerisation of business.  I discussed my belief that companies will be forced to reform as a portfolio of systematically designed components with clear outcomes and how this kind of thinking changes the relationship between a business capability and its IT support.  In particular I discussed the need to create industrialised Service Delivery Platforms which vastly increase the speed, reliability and cost effectiveness of delivering service realisations. 

In this post I’ll to move into the second part of the story where I’ll look more specifically at how we can realise the industrialisation of service delivery through the creation of an SDP.

Industrialisation 101

There has been a great deal written about industrialisation over the last few years and most of this literature has focused on IT infrastructure (i.e. hardware) where components and techniques are more commoditised.  As an example many of my Japanese colleagues have spent decades working with leaders in the automotive industry and experienced firsthand the techniques and processes used in zero defect manufacturing and the application of lean principles. Sharing this same mindset around reliabliity, zero defect and technology commoditisation they created a process for delivering reliable and guaranteed outcomes through pre-integration and testing of combinations of hardware and software.  This kind of infrastructure industrialisation enables much higher success rates whilst simultaneously reducing the costs and lead times of implementation. 

In order to explore this a little further and to set some context, let’s Just think for a moment about the way in which IT has traditionally served its business customers.  

We can see that generally speaking we are set a problem to solve and we then take a list of products selected by the customer – or often by one of our architects applying personal preference – and we try to integrate them together on the customers site, at the customers risk and at the customers expense. The problem is that we may never have used this particular combination of hardware, operating systems and middleware before – a problem that worsens exponentially as we increase the complexity of the solution, by the way – and so there are often glitches in their integration, it’s unclear how to manage them and there can’t be any guarantees about how they will perform when the whole thing is finally working. As a result projects take longer than they should – because much has to be learned from scratch every time – they cost a lot more than they should – because there are longer lead times to get things integrated, to get them working and then to get them into management – and, most damningly, they are often unreliable as there can be no guarantees that the combination will continue to work and there is learning needed to understand how to keep them up and running.

The idea of infrastructure industrialisation, however, helps us to concentrate on the technical capability required – do you want a Java application server? Well here it is, pre-integrated on known combinations of hardware and software and with manageability built in but – most importantly – tested to destruction with reference applications so that we can place some guarantees around the way this combination will perform in production.  As an example, 60% of the time taken within Fujitsu’s industrialisation process is in testing.  The whole idea of industrialisation is to transfer the risk to the provider – whether an internal IT department or an external provider – so that we are able to produce consistent results with standardised form and function, leading to quicker, more cost effective and reliable solutions for our customers.

Now such industrialisation has slowly been been maturing over the last few years but – as I stated at the beginning – has largely concentrated on infrastructure templating – hardware, operating systems and middleware combined and ready to receive applications.  Recent advances in virtualisation are also accelerating the commoditisation and industrialisation of IT infrastructure by making this templating process easier and more flexible than ever before.  Such industrialisation provides us with more reliable technology but does not address the ways in which we can realise higher level business value more rapidly and reliably.  The next (and more complex) challenge, therefore, is to take these same principles and apply them to the broader area of business service realisation and delivery.  The question is how we can do this?

Industrialisation From Top to Bottom

Well the first thing to do is understand how you are going to get from your expression of intent – i.e. the capability definitions I discussed in my previous post that abstract us away from implementation concerns – through to a running set of services that realise this capability on an industrialised Service Delivery Platform. This is a critical concern since If you don’t understand your end to end process then you can’t industrialise it through templating, transformation and automation.

In this context we can look at our capability definitions and map concepts in the business architecture model down to classifications in the service model.  Capabilities map to concrete services, macro processes map to orchestrations, people tasks map to workflows, top level metrics become SLAs to be managed etc. The service model essentially bridges the gap between the expression of intent described by the target business architecture and the physical reality of assets needed to execute within the technology environment.

From here we broadly need to understand how each of our service types will be realised in the physical environment – so for instance we need a physical host to receive and execute each type of service, we need to understand how SLAs are provisioned so that we can monitor them etc. etc.

Basically the concern at this stage is to understand the end to end process through which we will transform the data that we capture at each stage of the process into ever more concrete terms – all the way from logical expressions of intent through greater information about the messages, service levels and type of implementation required, through to a whole set of assets that are physically deployed and executing on the physical service platform, thus realising the intent.

The core aim of this process must be to maximise both standardisation of approach and automation at each stage to ensure repeatability and reliability of outcome – essentially our aim in this process is to give business capability owners much greater reliability and rapidity of outcome as they look to realise business value.  We essentially want to give guarantees that we can not only realise functionality rapidly but also that these realisations will execute reliably and at low cost.  In addition we must also ensure that the linkage between each level of abstraction remains in place so that information about running physical services can be used to judge the performance of the capability that they realise, maximising the levers of change available to the organisation by putting them in control of the facts and allowing them to ‘know sooner’ what is actually happening.

Having an end to end view of this process essentially creates the rough outline of the production line that needs to be created to realise value – it gives us a feel for the overall requirements.  Unfortunately, however, that’s the nice bit, the kind of bit that I like to do. Whilst we need to understand broadly how we envisage an end to end capability realisation process working, the real work is in the nasty bit – when it comes to industrialisation work has to start at the bottom.

Industrialisation from Bottom to Top

If you imagine the creation of a production line for any kind of physical good they obviously have to be designed to optimise the creation of the end product. Every little conveyer belt or twisty robot arm has to be calibrated to nudge or weld the item in exactly the same spot to achieve repeatability of outcome. In the same way any attempt to industrialise the process of capability realisation has to start at the bottom with a consideration of the environment within which the final physical assets will execute and of how to create assets optimised for this environment as efficiently as possible. I use a simple ‘industrialisation pyramid’ to visualise this concept, since increasingly specialised and high value automation and industrialisation needs to be built on broader and more generic industrialised foundations. In reality the process is actually highly iterative as you need to continually be recalibrating both up and down the hierarchy to ensure that the process is both efficient and realises the expressed intent but for the sake of simplicity you can assume that we just build this up from the bottom.

So let’s start at the bottom with the core infrastructure technologies – what are the physical hosts that are required to support service execution? What physical assets will services need to create in order to execute on top of them? How does each host combine together to provide the necessary broad infrastructure and what quality of service guarantees can we put around each kind of host? Slightly more broadly, how will we manage each of the infrastructure assets? This stage requires a broad range of activity not just to standardise and templatise the hosts themselves but also to aggregate them into a platform and to create all of the information standards and process that deployed services will need to conform to so that we can find, provision, run and manage them successfully.

Moving up the pyramid we can now start to think in more conceptual terms about the reference architecture that we want to impose – the service classifications we want to use, the patterns and practices we want to impose on the realisation of each type, and more specifically the development practices.  Importantly we need to be clear about how these service classifications map seamlessly onto the infrastructure hosting templates and lower level management standards to ensure that our patterns and practices are optimised – its only in this way that we can guarantee outcomes by streamlining the realisation and asset creation process. Gradually through this definition activity we begin to build up a metamodel of the types of assets that need to be created as we move from the conceptual to the physical and the links and transformations between them. This is absolutely key as it enables us to move to the next level – which I call automating the “means of production”.

This level becomes the production line that pushes us reliably and repeatably from capability definition through to physical realisation. The metamodel we built up in the previous tier helps us to define domain specific languages that simplify the process of generating the final output, allowing the capture of data about each asset and the background generation of code that conforms to our preferred classification structure, architectural patterns and development practices. These DSLs can then be pulled together into “factories” specialised to the realisation of each type of asset, with each DSL representing a different viewpoint for the particular capability in hand.  Individual factories can then be aggregated into a ‘capability realisation factory’ that drives the end to end process.  As I stated in my previous post the whole factory and DSL space is mildly controversial at the moment with Microsoft advocating explicit DSL and factory technologies and others continuing to work towards MDA or flexible open source alternatives.  It is suffice to say in this context that the approaches I’m advocating are possible via either model – a subject I might return to actually with some examples of each (for an excellent consideration of this whole area consult Martin Fowler’s great coverage, btw).

The final level of this pyramid is to actually start taking the capability realisation factories and tailoring them for the creation of industry specific offerings – perhaps a whole set of ‘factories’ around banking, retail or travel capabilities. From my perspective this is the furthest out and may actually not come to pass; despite Jack Greenfield’s compelling arguments I feel that the rise of SOA and SaaS will obviate the need to generate the same application many times by allowing the composing of solutions from shared utilities.  I feel that the idea of an application or service specific factory assumes a continuation of IT oversupply through many deployments; as a result I feel that the key issue at stake in the industrialisation arena is actually that of democratising access to the means of capability production by giving people the tools to create new value rapidly and reliably.  As a result I feel that improving the reliability and repeatability of capability realisation across the board is more critical than a focus on any particular industry. (This may change in future with demand, however, and one potential area of interest is industry specific composition factories rather than industry specific application generation factories). 

Delivering Industrialised Services

So we come at last to a picture that demonstrates how the various components of our approach come together from a high level process perspective.

Across the top we have our service factory. We start on the left hand side with capability modelling, capturing the metadata that describes the capability and what it is meant to do. In this context we can use a domain specific language that allows us to model capabilities explicitly within the tooling. Our aim is then to use the metadata captured about a capability to realise it as one or more services. In this context information from the metamodel is transformed into an initial version of the service before we use a service domain language to add further detail about contracts, messages and service levels. It is important to note that at this point, however, the service is still abstract – we have not bound it to any particular realisation strategy. Once we have designed the service in the abstract we can then choose an implementation strategy – example classifications could be interaction services for Uis, workflow services for people tasks, process services for service orchestrations, domain services for services that manage and manipulate data and integration services that allow adaptation and integration with legacy or external systems.

Once we have chosen a realisation strategy all of the metadata captured about the service is used to generate a partially populated realisation of the chosen type – in this context we anticipate having a factory for each kind of service that will control the patterns and practices used and provide guidance in context to the developer.

Once we have designed our services we now want to be able to design a virtual deployment environment for them based wholly on industrialised infrastructure templates. In this view we can configure and soft test the resources required to run our services before generating provisioning information that can be used to create the virtual environment needed to host the services.

In the service platform the provisioning information can be used to create a number of hosting engines, deploy the services into them, provision the infrastructure to run them and then set up the necessary monitoring before finally publishing them into a catalogue. The Service Platform therefore consists of a number of specialised infrastructure hosts supporting runtime execution, along with runtime services that provide – for example – provisioning and eventing support.

The final component of the platform is what I call a ‘service wrap’. This is an implementation of the ITSM disciplines tailored for our environment. In this context you will find the catalogue, service management, reporting and metering capabilities that are needed to manage the services at runtime (this is again a subset to make a point). In this space the service catalogue will bring together service metadata, reports about performance and usage plus subscription and onboarding processes.  Most importantly there is a strong link between the capabilities originally required and the services used to realise them, since both are linked in the catalogue to support business performance management. In this context we can see a feedback loop from the service wrap which enables capability owners to make decisions about effectiveness and rework their capabilities appropriately.

Summary

In this second post of three I have demonstrated how we can use the increasing power of abstraction delivered by service-orientation to drive the industrialisation of capability realisation.  Despite current initiatives broadly targeting the infrastructure space I have discussed my belief that full industrialisation across the infrastructure, applications, service and business domains requires the creation and consistent application of known patterns, processes, infrastructures and skills to increase repeatability and reliability. We might sacrifice some flexibility in technology choice or systems design but increasing commoditisation of technology makes this far less important than cost effectiveness and reliability. It’s particularly important to realise that when industrialising you need to understand your end to end process and then do the nasty bit – bottom up in excruciating detail.

So in the third and final post on this subject I’m going to look a little bit at futures and how the creation of standardised and commoditised service delivery platforms will affect the industry more broadly – essentially as technology becomes about access rather than ownership so we will see the rise of global service delivery platforms that support capability realisation and execution on behalf of many organisations.

Industrialised Service Delivery Redux I

I’ve been terribly lax with my posting of late due to pressures of work but thought I had best try and put something up just to keep my blog (barely) alive, lol.  Following on from my previous posts on Cloud Computing and Service Delivery Platforms I thought I would go the extra step and actually talk about my views on Industrialisation in the platform and service delivery spaces.  I made this grand decision since my last post included a reference to a presentation that I did in Redmond last year and so I thought it would be useful to actually tell the story as well as just punt up the slides (which can be pretty meaningless without a description).  In addition there’s also been a huge amount of coverage of both cloud computing, platform as a service and industrialisation lately and so it seemed like revisiting the content of that particular presentation would be a good idea.  If I’m honest I also have to admit that I can largely just rip the notes out of the slideset for a quick post, but I don’t feel too guilty given that it’s a hot topic, lol.  I’ll essentially split this story across three posts: part I will cover why I believe Industrialisation is critical to supporting agility and reliability in the new business environment, part II will cover my feelings on how we can approach the industrialisation of business service delivery and part III will look at the way in which industrialisation accelerates the shift to shared Service Delivery Platforms (or PaaS or utility computing or cloud computing – take your pick).

The Industrialisation Imperative

So why do I feel that IT industrialisation is so important?  Well essentially I believe that we’re on the verge of some huge changes in the IT industry and that we’re only just seeing the very earliest signs of these through the emergence of SOA, Web 2.0 and SaaS/PaaS. I believe that organisations are going to be forced to reform and disaggregate and that technology will become increasingly commoditised. Essentially I believe that we all need to recognise these trends and learn the lessons of industrialisation from other more mature industries – if we can’t begin to deliver IT that is rapid, reliable, cost effective and – most importantly – guaranteed to work then what hope is there?  IT has consistently failed to deliver expected value time and time again through an obsession with technology for it’s own sake and the years of cost overruns, late delivery and unreliability are well documented; too often projects seem to ignore the lessons of history and start from ground zero. This has got to change. Service orientation is allowing us to express IT in ways that are closer to the business than ever before, reducing the conceptual gap that has allowed IT to hide from censure behind complexity. Software as a Service is starting to prove that there are models that allow us to deliver the same function to many people with lower costs born of economies of scale and we’re all going to have to finally recognise that everyone is not special, that they don’t need that customisation or tailoring for 80% of what they do and that SOAs assistance in refocusing on business value will draw out the lunacy of many IT investments.

In this three part post I therefore want to share some of my ideas around how we can industrialise IT. Firstly, I’m going to talk about the forces that are acting on organisations that will drive increasing specialisation and disaggregation and go onto to discuss business capabilities and how they accelerate the commoditisation of IT.  Secondly, I’m going to discuss  approaches to the industrialisation of service delivery and look at the different levels of industrialisation that need to be considered.  Finally I’ll talk about how the increasing commoditisation and standardisation of IT will accelerate the process of platform consolidation and the resulting shift towards models that recognise the essentially scale based economics of IT platform provision.

The Componentisation of Business

Over the last 100 years we’ve seen a gradual shift towards concentration on smaller levels of business organisation due to the decreasing costs of executing transactions with 3rd parties. Continuing discontinuities around the web are sending these transaction costs into free fall, however, and we believe that this is going to trigger yet another reduction in business aggregation and cause us to focus on a smaller unit of business granularity – the capability (for an early post on this subject see here).

Essentially I believe that there are four major forces that will drive organisations to transform in this way:

1) Accelerating change;

2) Increasing commoditisation; and

3) Rapidly decreasing collaboration costs due to the emergence of the web as a viable global business network.

I’ll consider each in turn.

Accelerating Change

As the rate of change increases, so adaptability becomes a key requirement for survival. Most organisations are currently not well suited for this challenge, however, as they have structures carried over from a different age based on forward planning and command and control – they essentially focus inwards rather than outwards. The lack of systematic design in most organisations means that they rarely understand clearly how value is delivered and so cannot change effectively in response to external demand shifts. In order to become adaptable, however, organisations need to systematically understand what capabilities they need to satisfy demand and how these capabilities combine to deliver value – a systematic view enables us to understand the impact of change and to reconfigure our capabilities in response to shifts in external demand.

Increasing Commoditisation

This capability based view is also extremely important in addressing the shrinking commoditisation cycle. Essentially consumers are now able to instantly compare our goods and services with those from other companies – and switch just as quickly. A capability-based view enables us to ensure that we remove repetition and waste across organisational silos and replace these with shared capabilities to maximise our returns, both while the going is good but also when price sensitivity begins to bite.

Decreasing Transaction Costs

The final shift is to use our clearer view of the capabilities we need to begin thinking about those that are truly differentiating – the market will be putting such pressure on us to excel that we will be driven to take advantage of falling transaction costs and the global nature of the web to replace our non-differentiating capabilities with those of specialised partners, simultaneously increasing our focus, improving our overall proposition and reducing costs.

As a result of these drivers we view business capabilities as a key concept in the way in which we need to approach the industrialisation of services.

Componentisation Through Business Capabilities

So I’ve talked a lot about capabilities – how do they enable us to react to the discontinuities that I’ve discussed? Well to address the issues of adaptability and understand which things we want to do and which we want to unbundle we really need a way of understanding what the component parts of our organisation are and what they do.

Traditionally organisations use business processes, organisational structures or IT architectures as a way of expressing organisational design – perhaps all three if they use an enterprise architecture method. The big problem with these views, however, is that they tell us very little about what the combined output actually is – what is the thing that is being done, the essential business component that is being realised? Yes I understand that there are some people doing stuff using IT but what does it all amount to? Even worse, these views of the business are all inherently unstable since they are expressions of how things get done at a point in time; as a result they change regularly and at different rates and therefore make trying to understand the organisation a bit like catching jelly – you might get lucky and hold it for a second but it’ll shift and slip out of your grasp. This means that leaders within the organisation lack a consistent decision making framework and see instead a constantly shifting mass of incomplete and inconsistent detail that make it impossible to make well reasoned strategic decisions.

Capabilities bring another level of abstraction to the table; they allow us to look at the stable, component parts of the organisation without worrying about how they work. This gives us the opportunity to concentrate systematically on what things the organisation needs to do – in terms of outputs and commitments – without concerning ourselves with the details of how these commitments will be realised. This enables enterprise leaders to concentrate on what is required whilst delegating implementation to managers or partners. Essentially they are an expression of intent and express strategy as structure. Capabilities are then realised by their owners using a combination of organisational structures, role design, business processes and technology – all of which come together to deliver to the necessary commitments.

In this particular example we see the capability from both the external and internal perspectives – from the perspective of the business designer and the consumer the capability is a discrete component that has a purpose – in this case enabling us to check credit histories – and a set of metrics – for simplicity we’ve included just service level, cost and channels. From the perspective of the capability owner, however, the capability consists of all of the different elements needed to realise the external commitments.

So how does a shift to capabilities affect the relationship between the organisation and its IT provision?

IT Follows Move from “How” to “What”

One of the big issues for us all is that a concentration on capabilities will begin to push technology to the bottom of the stack – essentially it becomes much more commoditised.

Capability owners will now have a much tighter scope in the form of a well defined purpose and set of metrics; this gives them greater clarity and leaves them able to look for rapid and cost effective realisation rather than a mismash of hardware, software or packages that they then need to turn into something that might eventually approximate to their need.  Furthermore the codification of their services will expose them far more clearly to the harsh realities of having to deliver well defined value to the rest of the organisation and they will no longer be able to ‘lose’ the time and cost of messing about with IT in the general noise of a less focused organisation.

As a result capability owners will be looking for two different things:

1) Is there anyone who can provide this capability to me externally to the level of performance that I need – for instance SaaS or BPU offering available on a usage or subscription basis; or

2) Failing that who can help me to realise my capability as rapidly, reliably and cost effectively as possible.

The competition is therefore increasingly going to move away from point technologies – which become increasingly irrelevant – and move towards the delivery of outcomes using a broad range of disciplines tightly integrated into a rapid capability realisation platform.

Such realisation platforms – which I have been calling Service Delivery Platforms to denote their holistic nature – require us to bring infrastructure, application, business and service management disciplines into an integrated, reliable and scalable platform for capability realisation, reflecting the fact that service delivery is actually an holistic discipline and not a technology issue. Most critically – at least from our perspective – this platform needs to be highly industrialised; built from repeatable, reliable and guaranteed components in the infrastructure, application, business and service dimensions to guarantee successful outcomes to our customers.

So what would a Service Delivery Platform actually look like?

A Service Delivery Platform

In this picture I’ve surfaced a subset of the capabilities that I believe are required in the creation of a service delivery platform suitable for enterprise use – I’m not being secretive, I just ran out of room and so had to jettison some stuff.

If we start at the bottom we can see that we need to have highly scalable and templatised infrastructure that allows us to provide capacity on demand to ensure that we can meet the scaling needs of capability owners as they start to offer their services both inside and outside the organisation.

Above this we have a host of runtime capabilities that are needed to manage services running within the environment – identity management, provisioning, monitoring to ensure that delivered services meet their service levels, metering to support various monetisation strategies both from our perspective and from the capability owners perspective, audit and non-repudiation and brokering to external services in order to keep tabs on their performance for contractual purposes.

Moving up we have a number of templatised hosting engines – essentially we need to break the service space down using a classification to ensure that we are able to address different kinds of services effectively. These templates are essentially virtual machines that have services deployed into them and which are then delivered on the virtualised hardware; essentially the infrastructure becomes part of the service to decouple services both from each other and physical space.

The top level in the centre area is what we call service enablement. In this tier we essentially have a whole host of services that make the environment workable – examples that we were able to fit in here are service catalogue, performance reporting, subscription management – the whole higher level structure that brings services into the wider environment in a consistent and consumable way.

Moving across the left we can see that in order to deliver services developers will need to have standardised and templatised shared development support environments to support collaboration, process enablement and asset management.

Across on the right we have operational support – this is where we place our ITIL/ISO 20000 service management processes and personnel to ensure that all services are treated as assets – tracked, managed, reported upon, capacity managed etc etc.

On the far right we have a business support set of capabilities that support customer queries about services, how much they’ve been charged and where we also manage partners, perform billing or carry out any certification activity if we want to create new templates for inclusion in the overall platform.

Finally across the top we have what I call the ‘service factory’ – a highly templatised modelling and development environment that drives people from a conceptual view of the capabilities to be realised down through a process of service design, realisation and deployment against a set of architectural and development patterns represented in DSLs.  These DSLs could be combinations of UML profiles, little languages or full DSLs implemented specifically for the service domain.

Summary

In this post I have discussed my views on the way in which businesses will be forced to componentise and specialise and how this kind of thinking changes the relationship between a business capability and its IT support.  I’ve also briefly highlighted some of the key features that would need to be present within an holistic and industrialised Service Delivery Platform in order to increase the speed, reliability and cost effectiveness of delivering service realisations.  In the next post I’ll to move into the second part of the story where I’ll look more specifically at realising the industrialisation of service delivery through the creation of an SDP.

Does the Cloud lead to homogenised enterprises?

Just a quick follow on from my last post about cloud computing/service delivery platforms/platform as a service/saas platforms etc etc as. I was intrigued by a Joe McKendrick post on the merits of the model.  Joe’s analysis was pretty balanced as usual but one of his disadvantages caught my eye as I disagreed with it strongly.  Essentially Joe suggests that there is a perception that “Enterprises leveraging Cloud computing may become homogenized — and lose the competitive advantage that may come from custom-built systems”.  As I’ve previously discussed the concept of cloud computing drives people to finally decide what business they are in; if they work in an end user organisation then they need to recognise that and start treating IT like a utility rather than a differentiator.  In this context, however, I would argue that rather than homogenising the enterprise it will have the opposite effect – if we get rid of our IT platform to the cloud (who cares if it is homogenised – it’s just a technology platform), consume our 80% of non-differentiating business capabilities as SaaS or BPU services from the cloud (who cares if these are homogenised – they don’t differentiate us) and then focus all of our attention on realising the remaining 20% of differentiating capabilities – as custom built systems – more rapidly and cheaply on our chosen cloud computing platform(s) then we will be maximising our advantage whilst losing none of the benefits.  It’s important to realise that just because we don’t own and operate the physical technology it doesn’t mean that we can’t own the services that run on it  – we can still develop our custom-built systems to realise our differentiating capabilities.  And given that this is the only bit of the whole value chain that’s actually important for us to customise I would argue that we are – in reality – less likely to end up homogenised since we are more focused on where our value really lies and therefore more able to maximise our differentiation from our competitors.