Direction for the SOA Wave

The Fourth Wave Barreling In

The impact of SOA, the fourth wave of enterprise IT, has gone way beyond the early adopters. Leading research reports show that the vast majority of CIOs and CTOs have formally explored, and in many cases begun implementing, pilot initiatives. Recent surveys conducted by InfoWorld, Yankee Group, Webservices.org and others show that over 75% of CIOs have investigated and are planning SOA investments in the next 12 to 18 months.

Despite the growing enthusiasm, SOA lacks many of the shared experiences, artifacts and patterns required for widespread and reliable IT adoption. Moreover, without a common language and industry blueprints, this wave, which promises benefits of intra- and inter-enterprise services reuse as well as process interoperability, will dissipate and eventually fizzle – ultimately only adding more custom logic and methodologies to the IT legacy. Directing the SOA wave so that it can live up to its potential requires user community leadership.

Read more about this at http://dev.bea.com/arch2arch/soa_wave.jsp

Recommended Approach to SOA

It is the best of times, it is the worst of times, it is the time of service-oriented architecture (SOA), it is a time of traditional development methodologies, it is a time when products have matured, it is a time when products don't exist yet, it is the season of the optimist, it is the season of pessimist. We have endless possibilities in front of us, but we are concerned about being irrelevant. We are going to achieve the state of nirvana, we are going directly the other way—in short, this is a huge opportunity for IT to demonstrate its true value.

Read more about this at http://dev2dev.bea.com/pub/a/2005/10/approaches_to_soa.html

SOA Stakeholders

Indentify the SOA stakeholders for crafting the right message tothem.

Copy of my post at: http://dev2dev.bea.com/blog/YogishPai/archive/2005/09/soa_stakeholder.html

The consistent feedback I have been receiving over the past few months is requesting for a consistent definition of SOA. The messaging shall wary based on the target audience and is key to getting buyin to adopt SOA enterprisewide.
I shall attempt to identify the key SOA stakeholders and shall post the proposed messaging is the subsequent blogs.

IT "Board of Directors" Just like every organizations has a Board of Directors, IT needs one too. Typically this function is performed by some key executives or by their representatives. The Board's objective is to set direction, approve initiatives/projects, resolve conflicts, etc. Some organizations refer to these boards as ISSC (IS Steering Committe), ITRB (IT Review Board), etc.

Chief Information Officer (CIO) Head of the IT organization - responsible for all aspects of IT.

Chief Technology Officer (CTO) Responsible for Enterprise Architecture within IT. This has been a recent trend for creating this position in IT organizations.

Program Management Office (PMO) The PMO is responsible for orchestrating the projects across the enterprise or LOB. Ensure that the standard process defined by the Enterprise Architects is followed by each of the project teams and is primary contact for all Project Managers for cross-functional activities.

Architects
Multiple categories of Architect
Enterprise Architects are responsible for defining the standards, proces, design paterns, identify new technology, etc.
Projects Architects are responsible for the design of a particular business solution/application.
Information/Data Architects are responsible for ensure that there is a consistent information/data approach & model accross the enterprise.

Business Sponsor is someone from the business who champions the applicaiton within the business and is ultimately responsible for ensuring that the projects is successfully adopt within the enterprise. This person is generally responsible for securing the funding, business transformation and is generally also an officer (VP or above) of the company.

Business Operations teams are resposible for defining and documenting the operations proccesses. Ensuring that the deinfed processes are followed within their area of responsibility and are also the key point of contact for the frontline person of that business unit. Example: Sales Operations is responsible for ensuring that all submitted orders by the sales is reviewed and accepted.

Archtiecture Steering Committee. There is a steering committee for every projects or program and there needs to be one for enterprise archtiecture too. The members of this committee are the IT Leadership Team and key stakeholders from business operations.

Shared Services Responsible for developing the shared services for the enterprise or LOB. Multiple models exists - dedicated shared services team, shared services being part of the Archtiecture team, shared services developed by each project team (like open source).

Project Managers Responsible for delivering the project on-time, under budget and could be either from business or IT. Best practices are to have one person responsible for project delivery but have seen model where a business PM is paired with the IT PM for getting the project delivered.

Data Center Operations Responsible for the data center and keep all the applications up and running.

Application Support Multiple models - Dedicated centralized team, dedicated application support team within a LOB, permanent project team with developers rotating to application support role, outsourced, etc.

Please note - the objetive is to identify the SOA stakeholders and may not have done justice to the job decriptions. Hope to spark up a discussion - so please feel free to add your comments. Expand on the job desciptions, etc.

Is ESB the same as an EAI?

The fundamental question that a lot of folks are asking is whether ESB (Enterprise Service Bus) the same (or the next Generation of) as EAI (Enterprise Application Integration). My response to this is no:

EAI is still appropriate wherever it makes sense (but I would not take this approach any more). The architecture approach is ADAPTOR-->TRANSFORMATION--->BUSINESS PROCESS -->TRANSFORMATION--->ADAPTOR which leverages EAI tool like TIBCO, WebMethods, WLI, etc. Is this truly SOA? No! - it is basically a point-to-point implementation on a Hub. The developer of the process need worry about all the protocol translation, routing, etc. and this is generally tightly coupled with the business process.

ESB on the other hand decouples the protocol translation, message/event routing, limited message validation, etc. and is targetted for use by IT Operations. Developers need not worry about all these details, which makes it faster to roll out new services. However, to make this a vaiable approach within an enterprise, there is definitely a greater emphasis on architcture.

The ESB also enables servcie abstraction at various levels, thereby enabling multiple teams to develop a particular service. In addition, with it's integration with the Service Registration and Management Capability it definitely makes it easier for IT Operations to track and provide the SLA requested by the business. The above diagram illustrates (Source: Paul Patrick - Chief Architect for Services Infrastructure, BEA Systems) how the ESB enables creating a grid within the enterprise - allowing each business unit / organization to grow at their own pace.

IT Engagement Model

One of the key ingrediance for success is clearly defining the roles and responsibilities within IT. There are multiple stake holders in IT with each doing their best to provide the highest level of support to the business. Most of the time this results in people stepping over each other - especially as there generally is a not a clear definition of everyone's task. Most of the project failures are due to the confusion is the definition of the roles between the PMO, Project Manager and the Enterprise Architects, resulting in resposiblities overlap and lack of making decisions. Gartner has produced an excellent job of defining the IT engagement model at a very high-level and is independent whether organizations adopt SOA or not.

SOA - Development Organization

The benefits of Service Oriented Architecture (SOA) are not only reducing the Total Cost of Ownership (TCO) by reusing service, streamlining business process, etc. - it also reduces the total number of resources required to adopt SOA enterprise wide. The SOA - Development Organization shall generally be identical to the deployment model and by distributing the tasks by the tiers IT organizations shall need fewer very skilled resources (Presentation).

The Architecture Framework lays the foundation for the enterprise and the Agile Project Management enables large organizations successfully manage and deploy SOA enterprise wide. Without adopting either - the traditional (structured) development methodology and process does not enable enterprises achieve the full benefit.

SOA - Focus Areas

Following are the 8 SOA Focus areas that Executives need to consider while adopting SOA.

1. Governance & Organization
2. Standard Bodies
3. Regulation & Compliance
4. Management (of Programs, Projects, Services, Infrastructure, etc.)
5. Security (could be considered as part of management - best broken out seperately)
6. Business Process
7. Development tools and Methodology
8. Deployment tools and Methodology

Based on my discussion with BEA there are 4 Organization patterns/models (listed in the white paper available at http://eudownload.bea.com/uk/events/soa/soa.pdf). These models could be considered as:

Centralized, Hierarchial, Federated & Partially Federated and these models are typically structured the same way the Enterprise is organized. For example is the business is highly-centralized - then so is IT. This structure would typically not change for adopting SOA across the enterprise. Forrester has an excellent research paper on this - which you may want to download from their site.

It is easier to achive the reuse benefits of services in a centralized model. The issues for non-centralized (most large enterprises) organization is the structure/process required to get the benefits of reuse. The missing solutiuon for this is a centralized Service Repository with a Google like search capability. To make this a productive tool - every project manager & business would have to comit to document and publish their objectives, process, requirements, design, etc. (all the sharable assets) into this repository. Such a respository will have to leverage technologies such as Search (Goole Appliance, Autonomy, Verity, etc.) & Conetent Management Systems (Documentum, Interwoven, FileNet, etc.). Ofcouse - Microsoft provides similar capabilities but have not reviewed their products.

Defining SOA

Service Oriented Architecture means different things to different folks. To an IT Architect, it means the overall enterprise archtiecture definition and process that enables IT to develop and deploy business capability rapidly. To the LOB-IT liaisons it means the governance, organization and the process for project/program management. For them SOA means defining the various business bulding blocks that could potentially be reused which shall help them reduce cost. And finally for the CIO - SOA is basically the IT Strategy for delivering business capability.

The Architect shall refer to vendor site such as BEA, IBM, Oracle, SAP, Microsoft, etc. for product informatiion and sites such as The Server Side, W3C, JavaSoft, OASIS, etc. for standards or technical information. The LOB-IT and CIO's would typically attend some CIO Forums or subscribe to some very specific journals, articles etc. made specifically for them by vendors or analysts.

Again - everyone is looking at SOA from their point of reference. The IT specific approach shall work to an extent and get good business benefits. However to achieve the entire benefits, SOA has to be the Business Operations Strategy for the enterprise. Ofcourse, someone would then comment that the term Architecture means it is directed toward the IT organization. Ok then - this has been wrongly named - so what? It has the buz, marketing, hype whatever you want to call it - and executives are willing to give it a try (and pull out of it immediately - if they even get a wiff of failure).

Intel has termed this correctly as Services Oriented Enterprise (SOE) and have broken it down into two layers - Services Oriented Infrastructure (servers running on Intel) and Services Oriented Architecture (Business solutions running on SOA Platform). Unlike Intel (who wants to sell more of their chips) I would expect infrastructure to include network, hardware and software that manages and provides the service execution environment (yes! the one very similar to the telecommunication platform). IT (and in a few years business) should be able to change business logic in this run time environment with no or minimal downtime.

To get there - business needs to invest in the infrastructure up front but they won't without understanding the expected benefits. The right (and the only way) to do this is to adopt and Enterprise wide Architecture Framework (or commonly known as a methodology). Starting right from the head of the business operations (COO/President with full knowledge of the CEO and the board) enterprises need to adopt a Framework for doing business. They need to have the same rigor in business, similar to the ones in the IT operations department. This requires both business and IT transformations. In order to do this both business and IT needs to develop or adopt a Framework, which defines their basic principles, objectives as well as define the process on how they plan to implement SOA within their organization. There are two ways of doing this - take time and define a custom one or adopt one that has already been put together. My recommendation would be to adopt frameworks like Zachman or TOGAF (I shall elaborate on this in my later sections). I am currently in the process of reviewing these frameworks and hope to have some comments soon.

One more thing - I also feel strongly that there is a need for IT organizations to create an alliance to provide clarity round SOA. It is important that such an organization be driven by the customers rather than vendors - so that we get products that we really want and not the products that the vendors want to sell us. Feel free to lookup the SOA Alliance web site for additional details.

Enterprise Reference Architecture

The above diagram illustrates the Enterprise Reference Architecture from an Enterprise Architect's (CTO's) point of view. An architect reviews the enterprise in layers starting from the network infrastructure layer all the way up to the business capabilities (proceses) and workbench (Portals) delivered to the business.

IT Architecture Matters! Services Oriented Architecture & Competitive Advantage

The following link is to my presentation at the Silicon Valley BEA User Group.
http://www.svbug.org/slides/adopt_soa.ppt

This presentation covers our first two generation of SOA adoption at BEA (IT). It does not cover the governance and organization details.