SOA:Many Things!

SOA: Many things

SOA is not one but many things! Recall the story of blind men and the elephant; each one comes up with a different interpretation. Our individual perceptions and role in an organization is not an adequate basis to describe the entire SOA beast. Agendas of individuals or organizations defining SOA make it even harder to fathom through the myriad. For instance, for a vendor with a product line of applications, SOA is just web services. For a technologist, it is an IT issue and all about SOAP. SOA is about legal contracts for the general counsel, or delivering a business capability for a CIO.

In short, SOA means many things to many people. Yet we are not blind and should not be constrained by a narrow view. Thus the overall definition of SOA must arise from collective nature of the enterprise, with input from its constituents.

Let’s consider a situation involving the return of unused goods. In this typical reverse logistics scenario, understanding of a service varies from technologists to non-technologists, each with different priorities. For a technologist, a service is a network-enabled software component, which provides a specific capability. For a non-technologist, a service is some form of offering supplied by an organization that receives merchandise return requests, sends directions for shipment, updates inventory and applies credit to the customer. A customer service representative will make sure that customer requests are understood and s/he gets the information for the return. Shipment and receiving will be your focus if your role is operations. Accounting will be more interested in applying appropriate credit to the customer. While each participant has a defined focus, overall goal is to establish a means of delivering value to the customer.

This complex situation involves Customer Relation Management, Supply-Chain Management, and Enterprise Resource Planning. SOA solves this complexity by

• Decomposition
• Aggregation
• Orchestration

SOA decomposes this business process into well defined discrete services: some are automatic, some are manual, some short-lived and near real-time, some are long-running requiring human-intervention. Services may be aggregated and then orchestrated to create additional services depending on the granularity level of decomposition.

After all, a service is a unit of work done by one entity in order to meet the need of another entity. The primary entities in SOA are the service provider and the service consumer. At any point, the roles may be reversed, consumers becoming providers and vice versa. The consumer enjoys the value created by service without the ownership of the specific costs and benefits.

Service interactions, however, must be defined unambiguously for both entities to successfully communicate. Each interaction between the participants must be self-contained in a service contract, focusing on a specific capability. A greater synergy is achieved by the orchestration of these services in a coherent manner.

SOA embraces the fact that each entity may have a drastically different environment. Distinct systems or applications deployed at various agencies can interoperate because SOA makes as its pinnacle the use of ubiquitous XML which poses no limitation by platform or language. Choosing JAVA, .NET, or a mixture is not a headache anymore! By placing emphasis on business activity rather than implementation details, SOA can make that crucial inventory drop arrive at a warehouse manager’s Blackberry™ to shore up just-in-time measures. By adhering to the service contract, an organization can easily swap out its implementations or move to other locations. A service for sending out information to customers, for instance, may be reused by multiple departments.

SOA is more than a buzzword. It is the way for organizations to rapidly develop capabilities, to collaborate and meet the expectations of their stakeholders. SOA’s agility makes it an enterprise business strategy, however.

SOA may also be DOA, dead on arrival, if we narrowly assume that it is just a bunch of web services, a product or a panacea for integration nightmares.

Some organizations get their feet wet with a few services by creating prototypes and exploratory projects. This is a good point of entry for the technologists but the business executives need to start thinking big at the onset. For businesses, a concerted effort needs to be made to launch a SOA lifecycle at the enterprise or the segment level. A typical SOA lifecycle includes three stages:

• Initiation
• Roadmap development
• Plan execution

For continued success, the lessons learned in execution should create a feedback loop to the roadmap.

SOA initiation needs both business and IT to cooperatively decide on the business process that will be built or replaced. Creating the team and setting a project plan is necessary. Roadmap development establishes the SOA principles and reference architecture for business and technology (application, data, and infrastructure). Governance will guide with policies, measure effectiveness, control for compliance, communicate to inform all parties, and above all, empower for better decision making. Program management assures that the project is on time, on budget and on scope.

The SOA lifecycle is about implementing SOA enterprise-wide. It gauges smaller pilot projects to be on par to address the need of the organization and paves the way for properly scaling up. Once business and IT decide on the services, avoiding DOA requires establishing a Service Lifecycle to manage individual services.

Service lifecycle spans the activities from inception to retirement or decommissioning of a service. There are three stages:

• Requirements and Analysis
• Design and Development
• IT Operations

Following a service lifecycle assures accurate capture of business requirements, development of IT solutions that match business requirements and deployment/maintenance of services that provide business capabilities encapsulated in the requirements.

Starting a SOA lifecycle, or for that matter, choosing the candidate business processes along with governance, is not a small feat. A misfiring engine can cause headaches in the long drive. Choosing a vendor in this endeavor may be the biggest investment an organization may have to make in climbing the mount SOA.

Note that SOA is neither a product nor a single solution, but rather a comprehensive business enablement with IT elements. Make sure that your vendor is capable of bringing together business and IT stakeholders in a face-to-face collaborative where you, as an organization, not the vendor, make the decisions. After all, who knows your organization better than you? How rapidly will your vendor comprehend and assess your business processes to enable you to sail into service decomposition? Think big and ask about the end game. Establish governance and a program management. Spot the need to change early, especially organizational changes. Start small with services representative of your environment then you are ready to scale fast!

Burc Oral

http://www.linkedin.com/in/burcoral
http://www.soa-consortium.org/

SOA Consortium: Keynote by Amit Sinha (plus 2nd day comments)

Amit Sinha, VP Portfolio Marking from SAP presented the key note on the second day of the SOA Consortium face to face meeting held last week at Burlingame, CA. The topic he presented was on Business Network Transformation and following are some highlights from his presentation:



Most of these observations were based on the survey of over 175 CIOs globally. The trend has been that IT organizations are transformation themselves from Operational Excellence to Business Agility which in tech speak would be from Integrated Enterprises to Business Networks. Instead of developing solution based on the "Built to Last", the IT solutions now being considered (funded) are being "Built to Adapt".



Following are the top three approaches adopted by the various IT Organizations:

1. Managing Relationship (Single view of the customer, customer focused solutions, etc.). 23% of those surveyed adopted this approach which resulted in approximately 7x growth / year
2. Cost Containment (IT operational excellence, Off-shoring, BPO, etc.) 23% and the results are well know in the industry
3. Business Process Innovation (change the business model) 14% surveyed are adopting this approach and are consistently observing 2x the growth compared to the industry

In short - SOA is actually a catalyst for Innovation.

Following the key note we spent the rest of the day discussing the SOA Planning Framework, SOA Center Of Excellence as well SOA Skill sets required. One consistent theme throughout was that there is a desperate need of people with excellent Business breadth and deep technology skills - also referred to as the T shared person by Sandy Carter (IBM).

An observation while reviewing the EA 2010 presentation:

Improving IT Efficiency: 2% improvement to the margin

Going through Business Transformation: 8% improvement to the margin

Adopting SOA that facilities both Business and IT transformation: 20% improvement to the margin

Having an Enterprise Architecture team with both Business and Technology Architecture Skills: Priceless

- Yogish

Best Practices - Is Canonical XML dying? - Reply to IC Blog

This posting is a reply to the blog posting on Canonical Model http://www.icmembers.org/blog/zaki/2007/10/5/is-canonical-xml-dying#comments



In general, I agree with the posting on Canonical XML discussion. I wanted to make a couple of additional comments on industry-vertical specific Canonical models and the benefits they offer to those enterprises whose interactions leverage standards based information exchange.



a) implementations leveraging package products that exchange canonical models expressed in XML allow ease of upgrading and/or switching between package products without paying a huge upgrade penalty. This is so in the case of functionality related upgrades that do not alter the service definition and/or the information exchanged. Also, this makes decommisioning of redundant packages easy when this becomes necessary during acquisitions long as the package implementation are all standard canonical XML based.



b) partner collaborations also become easy, reliable and robust when collboration is based on the use of common industry vertical XML representations. Adding more trading partners becomes easy when using standards based information representations and this in turn leads to predicatable interaction patterns and information exchange.



c) documentation that refers to common service interactions that form the basis of that industry vertical when captured in training artifacts enable standardized interpretation of the material. In addition, given that this type of training is transferable between enterprises belonging to that industry vertical sharing personnel among trading partners is also made simpler.

Your comments on the canonical model usage is appreciated.
surekha -

SOA Consortium: Keynote by Sandy Carter

The SOA Consortium's last 2 days face to face meeting for 2007 is currently underway at Burlingame, CA. Yesterday, Sandy Carter gave a keynote address to the attendees and following is a brief summary of her comments:

Every two years, IBM conducts an exhaustive CEO survey (a couple of hours conversation with each of the CEOs) which is published every two years (last years report is available here). The next report shall be published next year around April at their IMPACT event. Following are some of the interesting findings:

Business have notices a 2% improvement in their revenue by focusing on IT optimization and an 8% improvement by focusing on business optimization. However, when they align both IT and Business, the impact was more than double, 20% a key reason for enterprises to focus on Business Agility.

The top four areas of focus for CEOs (in no particular order) are:

a) Agility
b) New and changing customer
c) Business Model Innovation
d) SOA (Yes! looks like SOA had now made the CEOs priority list)

As there is no real way of measuring agility, IBM has coined a new term KAIs (Key Agility Indicator). This is similar to KPIs (Key Performance Indicators) and they do have a tool that allows to benchmark your results to the 12,000 company data that they have compiled.

I like the concept of the KAIs and agree that what IBM had done is great job putting together a tool that allows enterprises to benchmark themselves against the industry. However, I believe that there is a need for a simpler way to measure your own agility (an art and a science) and it for this reason, I had taken a crack at the Business Agility domain model. Hope to define the maturity levels for each of the Business Agility domain model to enable business that do not have access to IBM's KAI tool.

As for new and changing customers - I agree things are changing at such a rapid pace, it is difficult to keep up, especially with the new social networks tool which shall rapidly enter the enterprise market. Hey! I just looked at Loopt and realized one could use such as tool for logistics and asset tracking.

Business Model Innovation is happening now, especially in large enterprises. Examples: Cisco moved from direct sales model to channel sales model. Oracle's model now has changed their business model from being a technology leader to an aggressive acquirer. And of-course we have companies like Google, Facebook and LinkedIn that have changed the way we search and collaborate.

SOA Consortium – Ground Floor SOA – Recap for 07

The most significant progress we on the SOA Consortium Working Group of "Executing Business Driven SOA" have made in the past year was in the definition of the SOA Planning Framework. Efforts undertaken as part of the SOA Planning Framework track will include classification and logical grouping of the collective body of work done thus far by the Strategy 2 working group; thereby allowing us to deliver a single document that can be used as a reference by anyone who embarks on the path to SOA-enable their enterprise. This includes work done under the headings of “SOA Readiness Assessment”, “SOA Opportunity Identification” and “High Level People, Process and Tools” matrix. All of this material will find a home in the various sub-components of the SOA Planning Framework. Once completed the SOA Planning Framework will be one of the most comprehensive guidelines for those looking to implement SOA.

The other key benefit of the SOA Planning Framework deliverable is that it allows an enterprise to start on one or more of these components without forcing it into having to define all components prior to getting started on the SOA Roadmap. This is where the SOA Planning Framework differs from that of a rigid SOA methodology - in that it is a loosely-woven collection of SOA enabling constructs.


Again, I would like to thank you of the members of Strategy 2 for your participation and invaluable input. I look forward to a very productive upcoming year.

surekha -

Selecting Strategic Projects

Continuing on my thoughts on aligning project portfolio to IT strategy, what are some of the criteria one must use for selecting projects that are truly strategic. A number of organization may use the size of the project to tag it as a strategic project. In my opinion, one must ask following questions to see if a project fits the definition of a strategic project:



- Is the project adding new capability or enhancing an existing capability in one of the core business processes. A core business process is one related to enterprises' primary value chain (excluding, HR, Finance, Facilities etc.).

- Is the capability shareable across many processes. For example a capability to have a full view of customer relationships can enhance not only sales, marketing and customer service processes but can also be used in other key processes such as supply chain management.

This is all I can think of right now. Would be interested in hearing other thoughts

Ashok Kumar

Key Lessons – The new and improved Enterprise Architecture Group

When invited to start the Enterprise Architecture Group a couple years ago my only thoughts were how to technologically implement SOA and EDA as the core architectural principles. However, soon enough came the realization that without having access to those who had sound business knowledge and those that knew the core business domains, value streams and the strategic business capabilities it would not be possible to effectively deliver SOA or EDA.


Service orientation of business behavior encapsulated in business applications and the definition of business events that initiate key business processes both require enterprise-wide understanding of business behaviors and business activities. Furthermore, the ability for identifying business services and/or business events comes from being able to abstract business needs that cross business domain boundaries. These skills can be found in a group of individuals being called business architects – a unique group of individuals who can translate the strategic business needs into an action plans that show how to enhance current business processes that support core value streams.


This led me to find resources that could analyze business processes (a.k.a. Process Architecture discipline) and resources that could analyze business information-flow, the value-addition pathways (where by business information is enhanced as it is exchanged across multiple business domains) (a.k.a. Information Architecture discipline). Both sets of these resources have allowed me to embark on true SOA style service definition. So the “new and improved” Enterprise Architecture Group has Process Architecture, Information Architecture (or what the industry calls Business Architecture) and Technical Architecture representatives.


Finally a note about the Technical Architects; my opinion is that, standards are vital but then if the Enterprise Architecture Group is unable to demonstrate the right use of standards in a true production environment this group will be left with very little credibility or be deemed the “ivory tower” group.


Having a cross-functional group of resources has made the Enterprise Architecture Group to be a well rounded group. The Business Architects of the group are in dialogue with the business as they can speak to the business in its lingo while the Technical Architects of the group are able to work with the developers as they are able to demonstrate the use of technology for building and delivering stable, performant and reusable services. The end result is that the individuals of Enterprise Architecture Group are openly invited to the table for consultations and recommendations.


I would love to hear of other organizational patterns/ formats that have enabled implementation of SOA and EDA.

Thanks.
surekha -

Integration at Home - SOA Architecture

An article describes how integration at home works - integrating PCs, Game consoles, storage, etc. as well as handling multi-media and home automation. This same approach is also very relevant to the services approaches for enterprises. Read more about this iin my this month's DMReview column on Integration at Home - SOA Architecture.

For those in the Silicon Valley - DAMA and the Integration Consortium (IC) have joined forces to bring together their respective organizations to offer a 1 day meeting to explore Data Management & Integration. Presentations from thought leaders from each organization. Additional details are available on IC web site at http://icdama.icmembers.org/.



- Yogish

Aligning Project Portfolio to IT Strategy

How an enterprise derives value from its IT organization greatly depends on the way IT manages its project portfolio. The projects should be funded, based on a well defined strategic direction that is in line with long term business objectives. The reality is that majority of projects are primarily undertaken to alleviate a major pain point or to deliver some functionality quickly to gain some competitive advantage or worse as a result of some internal politics. This problem is specially acute in organizations with large legacy systems where there is built in resistance to change and general tendency to be risk averse.

In my opinion, IT organizations must make a deliberate effort to make sure a percentage of IT budget is invested in projects that are strategic in nature. This also implies that both business and IT strategies exist and IT and business community have a common understanding of what it is and have a deep commitment for its success.

Blueprinting Information Architecture and BPM

Following my blog on Key Learning: Blueprinting Information Architecture is key to successful adoption of SOA one of the feedback was Don't we need to define the detailed business process to identify potential service candidates?

My response to that is No and explained it in this edition of the Strategic IT Update. Please click here for all the Strategic IT Update presentation and transcripts.

Reference: Customer Data Integration: BEA-IT Case Study (my presentation at CDI Summit in 2006)

Best Practices - Mashups and the importance of Foundation Business Services

It seems like interactive web based social computing experience and SOA style services are not really thought of as related. However, when mashups are incorporated into the realm of the enterprise the need for ease of collaboration as well as accuracy tend to matter and this is where enterprise worthy services enter the picture.

In addition, having access to a registry is key. This SOA based infrastructure component becomes the central point for discovering loosely-coupled services that encapsulate core business functions and business semantics. To fully enable the knowledge worker in the selection of services for mashups, the business user community not only has to be included in the definition of the service but it has to be involved with the process of publishing these services to the registry as well. The reason being that business keywords that are used in describing the service are identified and entered when publishing the service to the registry. This level of cooperation and collaboration with the business user insures that the services are described using business language and grammer.

Finally, cross-business service payload validation behavior has to be put in place to prevent users from creating mashups that combine incompatible services. For instance, only services sharing related business contexts or those that return information at the same level of granularity should be allowed to be pulled together. Tools/ IDEs of the future that aid in the creation of mashups may be able to interrogate the categorization of business oriented services and informational metadata descriptors published into a registry or a repository to make a determination of inter-service compatibility. In the near term however, an enterprise will have to resort to governanace and testing policies to allow compatible types of services to be mashed up.

As can be seen in an enterprise context users can gain the capability to perform effective mashups when they can leverage reliable foundation services to access enterprise information systems. This way IT can enable user self-service by just making core services available to the knowledge worker and enhance business agility.

Your feedback is invaluable.
surekha -

Strategic IT Update: Changing Role of C-Level Executives

The business environment is becoming more complex, unpredictable and constantly changing. In the near future IT will no longer be the way it is today. Business and Technology integration will be a necessity for business to stay competitive. To achieve this, the CFOs role is expected to become increasing important as described in this video.

Following is the first Strategic IT Update.

As this is my first production, I have intentionally kept it short so as to work out the release logistics.

Yogish Pai

Resolved RSS issues

Over the weekend I received feedback that the RSS feed on this site was not working. I believe that this has now been fixed and you can be subscribed to it once again at http://feeds.feedburner.com/AdoptingServiceOrientedArchitecture.

Please do drop me a line at info@soablueprint.com if you still having problems subscribing to the feeds.

Key Observations - Service Reusability - Does it really work?

From my experience, on SOA projects reuse was mostly accomplished with technical services/ infrastructure services. However, it is the business service reuse that we are struggling with as the Lines of Business (LoB)/ business units feel that their sub-set of the business logic is very unique. The problem is that most of these business services are rendered non-reusable due to the fact that they include both business logic and process sequencing logic into the same code base and it is the later that is unique to the LoB and not the business logic. Also, we have achieved greater degree of reusablitiy in terms of data access and informational services and not so much business function services that are a higher level class of services.

I would like to hear about what your experience has been with regards to service reusablity. Have you been able accomplish this.

Thanks,
surekha -

Defining and measuring Business Agility

In my previous post on defining Business Agility I had defined it as follows

Business Agility = Business Alignment + IT Flexibility

Business Alignment: Alignment between the various business units
IT Flexibility: Alignment between Business and IT + ability to rapidly deploy new business capability

Business Agility = Business Intuition + IT * (BPM + SOA)

Of course this definition is pretty generic and the next obvious question is how do we measure Business Agility? Typically most business have one primary measurement criteria - company
financial results such as revenue, operations cost, stock price and profits. There are alternate ways for measure business by adopting Six Sigma, Lean, etc. Again this approach is still business focused - and does not help in measuring IT Flexibility (especially as very rarely have I see IT organizations actually adopt Lean, Six Sigma or any such measuring criteria - maybe that will change :) ).

After giving this much thought I came up with this domain model for measuring Business Agility.

The Business Agility would be measured against each of the following domains:

Business Strategy (Business):

• Business Strategy clearly defined and communicated
• Clear defined goals for objectives for each of the business units
• Responsibility and Accountability
• Enterprise culture
• Innovation capability

Operations Framework (Business):

• Alignment of the business unit objectives to the enterprise objectives
• Alignment of operations teams across business silos
• Accountability and measurement of delivery to enterprise goals, rather than business unit goals
• Global Business Process definitions with localization capabilities
• Politics and interaction between business units
• Major part of compensation liked to enterprise goals, rather than business unit goals.

Information Strategy (Business):

• Identified clearly information needs for supporting business process
• Identified common business entities/objects such as customers, products, orders and partners that span business processes and silos
• Classified data into reference, transactional, master and analytics
• Information Governance for managing common data across business units

Governance (Business & IT)

• Business Governance (details to follow later)
• IT Governance (details to follow later)

Branding (Business & IT)

• Business Branding (customer, web site, multi-channel, products, etc.)
• IT Branding (Business perception, customer/partner perception, IT staff, etc.)

Technology Strategy (IT)

• Developing IT Roadmap (SOA Blueprinting, Standards, Architecture, etc.)
• Current state (baseline)
• Future/Target state
• Technology Roadmap aligned to business priorities

Program Management (IT)

• Program Management Office
• Application life cycle
• Application Portfolio Management
• SOA life cycle
• IT Governance (SOA Governance)

IT Operations Maturity (IT)

• CMM Level
• ITIL adoption
• SLAs
• Business Continuity
• Infrastructure maturity

These are defined at a very board level and need to be drilled down to identify or develop a maturity model for each of these domains.

- Yogish -

Best Practices - SOA Guiding Principles

From my experience, with SOA Retail solutions the following SOA Guiding Principles have served us well. We have applied this "standard" to both internal SOA style business services and when evaluating other packaged products that claim to be "SOA Enabled"!!!

a) business service consumers are decoupled from business service providers.
b) availability of published "service metadata" enables the discovery of the business service.
c) service consumers are agnostic of who (service provider location) and how (service provider technology platform) the function call is being fulfilled.
d) service consumers and service providers that use industry standard protocols and communications models are able to insure loose coupling.
e) service interaction is based upon exchange of canonical models

Please provide me with your feedback.
surekha -

Best Practice: SOA Development Organization

In early 2006 I had documented my views on how the IT development organizations would change by adopting SOA. I had termed it the SOA Development Organization and I still believe that this is the end state for the IT application development organization.

- Yogish -

Best Practices: Establishing Enterprise Architecture Teams

In one of my previous post I had published the first draft of the presentation on Establishing the Enterprise Architecture team. Based on some feedback and my own experience, I have now updated it with the following additional details:

1. The strategic role of the EA team
2. Enterprise Blueprinting benefits and guidelines
3. Enterprise Architecture Review process

and finally the EA domain model from the SOA Consortium EA 2010 effort. Please click here for the pdf version and do drop me a line at feedback@soablueprint.com for the PowerPoint version.

- Yogish -

Key Learnings - Living Business Driven SOA

There are some significant challeges in getting business buy-in on supporting SOA. Back in August my work in obtaining this support was recorded in the following interview with Jennifer Zaino of bITa Planet.

The opening statements of the interview are as follows:

"Some of the Words of advice from the frontlines: One IT leader discusses the challenges of getting business buy-in for service-orientated architecture (SOA)."

Please read the complete list of this conversation:

http://www.bitaplanet.com/alignment/article.php/3694846

I would love to hear you views on this matter.
surekha -

Key Learnings - Profile of a Business Service

My experience has led to the creation of the following service based postulates. These were applied by me on major retail SOA projects.

You can find these and other laws posted on the SOA Blueprint site. http://www.soablueprint.com/soa_laws

1. A service should expose a single business function or business process in it’s' entirety
2. A service should not make assumptions of the validity of the request and should validate the request for accuracy including checking that the requestor is authorized properly prior to honoring the call
3. A service should delegate to the right enterprise resources to completethe business function
4. A service should be responsible for notifying and dispatching the right requests/ information to all related services and/or applications on behalf of the requestor
5. A service should be responsible for any call chaining, sequencing and aggregating of the requests and the intermediate responses in order to completely process the consumer’s original call
6. A service should provide business information as a canonical documentthat is consumable by both human actors and system actors
7. A service should return a response that has the business context built in along with the response to the question asked by the requestor
8. A service should rely on enterprise worthy metadata information including the application of any algorithms, enterprise-wide business rules and strategies in order to process the requestors' call

Your feedback is appreciated.

- surekha -

Selling SOA to business

One of the most important milestone in SOA adoption is getting the business to buy in. However, getting business buy in requires an approach that is sometimes not obvious. I had an interesting conversation with Jon Brodkin of Network World. Please check it out at



http://www.networkworld.com/supp/2007/ndc6/102207-avis-soa-case-study.html



and let me know what you think





Ashok Kumar

Challenges in implementing BAM - drawing a relationship between business processes and operational system exceptions

I have a question to pose to those of you that have end to end business processes that are composed of multiple legacy business applications/ business services that may be deployed on disparate platforms and technologies. I am curious about what architecture patterns have been deployed by you to report on the operational state of business processes. Also, I would love to hear about how you provide business executives visibility to the business process execution related issues specifically to those metrics that impact business SLAs and KPIs governing the core value streams.


Product vendors have touted Business Activity Monitoring (BAM) to be the magic bullet that allows the creation of intuitive executive dashboards. Most BAM tools that monitor KPIs expect to subscribe to business events/ business alerts being generated from the business process. There in lies the challenge. The reality of where to gather the relevant data for creating the executive dashboard is what complicates most BAM implementations!!! The going in assumption is that these business applications are not really conducive to refactoring and may be mission critical systems that cannot be re-architected to follow an Event Driven Architecture EDA model that expresses it's internal state via a series of events.


In my experience most of the legacy business applications are created with both the business logic and the process logic embedded in them. In addition, most of these business applications do not really communicate or interact using business events/ business alerts. Furthermore, business applications that encapsulate parts of a business process may leverage messaging to communicate states to the next step in the business process. These business applications may have custom message listeners and handlers that transform the message and execute pertinent APIs that represent the next activity of the business process. Given this, I am wondering how folks are dealing with abstracting and discerning between the business exceptions, system exceptions and business process exceptions. All these categories of exceptions and the internal state of the business information may need to be monitored in order to aggregate information and accurately report the health of the business process on an executive dashboard.


We have resorted to the following architecture pattern. The various form of exceptions and business entity information (whether they be business or system or process related) are extracted from the logs, business application operational tables or system monitoring data structures. These data points or the heart beats across the process are then persisted into a business process state-monitoring data-mart. The extraction process has transformation and reconciliation logic to correlate the exceptions with the state of the business information. The extraction process also has the logic build to deal with the after effects of data discrepancies and business information inaccuracies caused by communications disruptions and other system exceptions. The baseline threshold latency between the steps in the process is used to then report back on how the system alerts or data integrity issues may have led to missing the SLA for a particular process. It must be noted that the ETL tool being used is an industry strength ETL tool and not some data and log scrapping scripts.


Finally , vendor BAM tools could be used but the key challenge lies in the ability to reconcile between which system issue or data integrity issues caused the business alert. I would love to hear about other common patterns that are being used to address this challenge of gathering data to compute BAM metrics!!

Best Practices: Establishing Enterprise Architecture Teams

Recently I have had a number of conversation with some key decision makers of various IT organizations and the common questions I got asked was to explain the process of establishing enterprise architecture teams.


This first requires the understanding the various IT organization patterns (models) as well as the evolution (maturity*) of the EA teams. My initial thoughts on this are available here.

Based on the feedback / comments from the community, I plan to take it to the next level of document the entry and exit criteria as well as the deliverable by the EA teams though each of phases.










* As there are already a lot of maturity models out there have decided to stay away from that term

Key Learnings: Blueprinting Information Architecture is key successful adoption of SOA

Following is the high-level approach for Blueprinting the SOA Framework for the enterprise.

Understand the business process and identify the areas for improvement
„Understand overall business objectives
„Understand specific business pain points
„Understand priorities

Develop the Enterprise Information Models to identify the Information services needed to support business processes

Develop the SOA Framework foundation to identify the infrastructure needed to support the business process needs.





Currently there are a lot of content available on the topic of best practices, key learning, reference architectures, ROI, etc. on the topics of Business Processes and Infrastructure. However there does not seem to be a lot of publications around Blueprinting the Information Architecture for SOA.

Basically there are three areas of focus for Blueprinting the Enterprise Information Architecture and are as follows:

• Reference data: Examples: salutation, job title, contact type, partner type, region, state/province, country, currency, language/locale, and industry
• Master data: Examples: customers, contacts, products and orders
• Analytic data: Examples: marketing data mart, forecasting, customer Life cycle value and customer satisfaction

Categorizing the key enterprise information needs into these three areas enables IT to have an very fruitful discussion with the business. However, the first task is to get business to fund and participate in this Blueprinting effort. Following is a very high level three step approach to Blueprinting the Information Architecture.

1. Scope and Planning (Establish: Budget and timeline, Team, Deliverable)
2. Information Architecture Assessment (Current state: high-level business process, data flow, data models - reference, master and analytics)
3. Information Architecture Blueprinting (Enterprise Information Model, Data Governance, Recommendations and Actionable Road map)

Why is this necessary?

Most business processes cross business and application silos. The current best practice is to migrate/move the data across these silos which results in redundant data, systems and places a lot of burden on business operations teams to clean it manually. In addition, multiple business silos tend to implement the same business process in different applications.

Adopting SOA enables enterprises to reduce the overall cost by enabling business to implement their business processes by consolidating/reusing a single infrastructure/application. This is possible only by exposing the appropriate information services to support the business processes.

This can be achieved only by truly understanding the Enterprise Information Model. Please click here for a three slide presentation on this.

Overhauled the SOA Blueprint site

The previous SOA Blueprint site was not very well organized and nor was it easy to maintain. As I was browsing through the Yahoo! Web Hosting Console I came across the "Site Solution" tool which makes it easier to manage and publish the content. Hope you like the new layout and please do feel free to send me your feedback, comments and/or any questions related to SOA.

Following are the categories of content currently posted on the site.

• SOA Practitioners Guide
• SOA Case Studies - reference to case studies published by forums, analysts and vendors
• SOA Organization and Governance
• Enterprise Architecture Frameworks - links to the most common EA frameworks
• Enterprise Architecture Resources - Resource page with information relevant to the Enterprise Architecture teams
• SOA Reference Architecture - Reference to a list of various published SOA Reference Architectures
• SOA Maturity Models - There are multiple maturity model and this page attempts to list all the major ones along with some comments, if possible.

Please feel free to also send relevant links or documents that would be useful for adopting SOA.

Enterprise Architecture Survey Results

In the SOA Consortium: Enterprise Architecture 2010 Webcast both Ashok and I refer to the Enterprise Architecture survey which was conducted earlier this year. This was an unscientific and informal survey conductedand for those interested the survey results are available here.

SOA Consortium: Enterprise Architecture 2010 Webcast

One of the working groups in the SOA Consortium’s community of practice is the “EA2010” group. This group of seasoned enterprise architects from industry, government, systems integrators and vendors, has been actively discussing and defining the next generation role of enterprise architecture. Specifically, what enterprise architecture looks like – organization, practices and people – in a business-driven, service-oriented world.

We are excited to share the first release of “Enterprise Architecture 2010 – Where Enterprise Architecture Means Business” in a slide deck now available for download.

Prior to the release of this important deliverable, Brenda Michelson, SOA Consortium Program Director, sat down with the EA2010 working group leaders, Ashok Kumar and Yogish Pai, to discuss their working group’s motivation, findings and next steps. Their conversation is available as an on-demand webinar.

In this 31-minute webinar, Ashok and Yogish cover a wide range of enterprise architecture concerns, including catalyzing business change, gaining business-smarts, shifting focus to business architecture, managing enterprise architecture, participating in strategy and delivery, and winning enterprise constituents.

Click here for more details on Enterprise Architecture 2010 Webcast.

Semantics and its' role in Business Services

Role and importance of semantics in the context of services and SOA:

Semantics refer to interpretation of information and not the literal definition of information/ data. Applying semantics to information turns it into “knowledge”. Semantics is the act of applying references and drawing conclusions given a set of more scientific informational constructs. Typically semantics are derived using the context in which information is presented. Transposition on the other hand allows applies the rule of inference where in one can draw conclusions on the implication of truth based on some set of facts.


By deduction semantics and semantic transposition deals with alternate inferences being drawn from one or more data and informational constructs using business rules. Inferences of this nature are not completely predictable as the rules used to arrive at the interpretation may or may not be quantifiable and/or may not be codified into a business rule.


The biggest risk in dealing with semantics is the lack of predictability of the outcome of an interaction, whether manual or automated. The reason is that the rules driving the decision tree or the execution path are not quantifiable.


Despite the complexity of interactions brought about by semantics and the complexity in being able to interpret these to codify and execute alternate behavioral patterns, one can attempt to capture metadata and leverage naming conventions to make the service based interaction more predictable. These guidelines may either be applied within an enterprise or could even have applicability to govern service interactions within an industry vertical.


In the realm of services, semantics drive both the discovery of services and also the execution paths and alternate process logic that is undertaken by the service provider. Naming conventions and expected service behavioral guidelines govern candidate service descriptions. Having documented behavioral implications and constraints associated with these naming conventions make the service “discovery” of a service and the expected interaction with the service provider more predictable from the service consumers perspective. These service naming conventions also act as metadata that can be used to publish information about the service definition during the publication and registry process.


Furthermore, one could make the assumption that there are certain features of a service action that denote the type of expected behavior, notwithstanding the type of business concept or the business function being manipulated by the service invocation. The service business action verb component can be associated with the general description of the type of behavioral patterns a consumer can expect. Given that SOA style services follow design by contract principles the naming and the signature of the service definition or description does not bind or affect the manner in which the service is implemented. This metadata of the verb does not impact communications patterns internally used by the service or even the informational model or informational exchange pathways that might internally be used by the service. These details are left up to the service provider. Having action verb nails expected behavioral semantics (inquiry only vs. alteration of service information model state) and not implementation semantics. Also, the action verb points to expected interaction semantics (one way/ two way/ guaranteed delivery etc.) and not the implementation model used by the service provider.


The key is that having these kinds of action-verb semantics outlined makes the service usage more specific. The service based action/ verb may be used by the consumer to identify the right service to use during the process of discovery. Also, the provider can use the action/verb notation to inform the consumer about the implications of the information being returned.


For example, a service operation called “obtainLoanRate()” with a action/verb embedded as the service invocation metadata may inform the consumer that the information being returned in not legally binding on the service provider. On the other hand, using the “lock” action/verb in the service invocation metadata “obtainLoanRate()” implies that the service provider is locking the rate and that the consumer is also bound legally to non-repudiation of the “rate” being locked. In this case, the semantics of the verb can be used by the consumer context to pick the right service depending on the usage context on the consumer tier. In addition the action/verb pairing may be used by the provider to insure that the right application logic is invoked. For instance, when the service consumer calls with a “get” action/verb invocation context the service provider implementation layer may go through fewer rigors and processing cycles than would be the case when the service consumer calls with a “lock” action/ verb invocation context.

Another example of the metadata driving semantics is the use of business context field by the service provider. A service operation “getCustomerProfile()” that uses the context of “Customer Profitability” or “Customer Credit Rating” could alter the information being returned in the payload of Customer Profiles. The service provider uses the service usage contextual information provided by the consumer to decide what type of information is returned to the consumer. The consumer context allows the service provider to alter the information returned between providing “Customer Profitability Range” or the “Customer Credit Score” in the customer profile.


As has been mentioned semantics can be applied to both the service publication and service discovery process. As part of the service publication process (to the registry) the service provider may use metadata in order to express the need for more precise contextual information to drive the way the service provider interprets the request. The service provider may add this constraint to leverage this information for altering the returned payload. The service consumer may also be able to use this business context metadata provided by the service provider along with the service provider business action-verb name to derive information about expected behavior of the service provider and how the request might be interpreted. For instance, a service operation that has an action-verb called “preview” may inform the service consumer that the information may not be reliable or even persisted beyond the current session while having a service consumer invoke a service operation with an action/verb “reserve” indicates to the consumer that the information returned is persisted beyond the current session. This naming convention is indicative of the service provider confidence factor in the consumption of the information being returned by the service provider.


In addition having some area for capturing the concept of the consumer context may help the service provider apply this knowledge to execute alternate paths of the business use case scenario. To some small extent, semantic transposition can be achieved by incorporating rules that have been gleaned from observing human behavior. If these manual workflows and best practices can be captured then these business rules can be used to execute alternate flows in the service. Consumer context information can then be used to invoke alternate business functionality and/or pathways that are known at compile time but may be loaded at runtime or invoked at run time on an as-needed basis.


The behavior altering ability can help the provider to honor consumer requests with more precision and with greater compliance to the contracted SLAs. This may also enable the service provider to roll out more behavior without altering the information models being exchanged and without altering the service interface. This may be a mechanism to curb the service version sprawl!!


Finally, the business context that is expected by the service provider or the business context information and candidate service name as published by the service provider could be classified as "service usage semantics". The purpose of the service usage semantics is to inform the service a consumer about the possibility of semantic variations in the honoring of service call.


Issues in incorporating Semantics in the realm of services and SOA


Semantics and "Semantic Transposition" may be fairly complicated to achieve in business systems. Most applications/services are typically written to fall cleanly into the “known” process logic pathways with no ability of either interpreting independent unrelated pieces of information. All paths typically have to be known and defined at compile time. In most situations, semantic interpretation may be complicated enough or unique enough to only be undertaken manually by knowledge workers. However, there may be a small subset of cases where the interpretations and inferences that drawn by the knowledge workers may be codified into rules and contextual metadata to help in the execution of alternate flows of a business use case.


Impact of semantics on Web Services, Service Registry and Service Repositories


As the concept of SOA and SOA style services become more standardized service related metadata registries/ repositories such as UDDI hosting web service URLs, service contract definitions and service descriptors defined as WSDL may make provisions to allow a service provider to publish service usage semantics, standard classifications and ontologies. These sets of metadata may become a guide in aiding the service consumers to browse for and find services more intelligently.


Furthermore, having a standardized mechanism to express the semantic usage information may enable tools to automatically bring this information back when service consumers are browsing for potential services. It must be noted that if this information is just some comment or a open ended description field instead of it being a structured metatag with known enumeration of values then browsing and searching for services cannot be automated.


Each industry vertical may have different values that might be populated into the standard structure to help with automating the search. These fields and valid industry vertical enumeration of values could all be published into the UDDI registry to help the consumer in either formulating the request or in informing the consumer about how to process the response.Some sample metatags could include fields such as business area field (Credit/ Sales/ Logistics).One such specification that is addressing Semantics in the realm of services is the Semantic Annotations for WSDL specification. http://www.w3.org/2002/ws/sawsdl/.


Conclusion


Having some stringency bound in using naming convention and in creating the request canonical models allows execution of the right sub-set of the service behavior thus making the service interaction more optimal. The presence of semantic oriented metadata allows the service provider to alter the behavior being executed in response to a service invocation and/or alters the amount and type of information returned to the service consumer.


Although not applicable in all situations, having contextual information and action/verb usage metadata information enables semantic transposition and execution of alternative service provider behavior s without having to resort to the execution of a human workflow and manual intervention. The objective of associating semantics to action verbs or in associating consumer context is not to bypass human intervention always but to use the learning obtained from the knowledge worker and encapsulate it in process automation logic.

Enterprise Architecture: Making yourself recruitable...

A interested blog by James McGovern on "Enterprise Architecture: Making yourself recruitable..." for those enterprise architects looking for their next career opportunity.

Key Learnings: SOA Key Success Factors (BEA-IT 2002-2006)

Based on my conversation with my peers in the industry, there is still a lot of keen desire by IT Leadership teams to understand the key success factors for adopting SOA. Following are the list of key success factors we (IT leadership team) had identified while I was the CTO-IT at BEA Systems.

1. Build the right team
2. Organize for success
3. Build coalition with business partners
4. Maintain Flexibility

The slides of these are available here. I have created a link of all my key learning blogs at my structured blog.

SOA Practitioners Guide Part 4: SOA and ITIL Convergence

The SOA Practitioners have been working on the next set of the Practitioners' Guide with SOA and ITIL Convergence being the first (Part 4) of new set of Guides. Burc Oral has been leading this effort for the Practitioners' and is presentation (SOA Practitioners Guide Part 4: SOA and ITIL Convergence) this at the eGov meeting today (October 1st, 2007).

Integration Tomorrow, Part 2: SOA Architecture

Part 1 of the two-part series described the past and current integration approaches. This second part, takes a practical approach on how technology is going to change business landscape and the potential infrastructure changes required to integrate the business both at business and IT levels.



Read more about this here.

Vendors need to incorporate SOA infrastructure in legay applications

The majority of the IT budget (over 80%) is typically committed to supporting the existing infrastructure and applications. Even though adopting SOA brings substantial value over the long term, the short-term impact actually increases the support cost in two ways.

First, IT organizations will need to procure additions infrastructure components such as the Enterprise Service Bus, Registry/Repository, SOA Manager, SOA Governance tools, etc. Products like these not only requires upfront initial investment, it also adds one more layer of abstraction (hop) in the existing environment impacting the end-to-end performance.

Second, the support organization shall need to increase headcount and hardware infrastructure to support this next generation of infrastructure.

Even though most of the existing packaged applications are re-architecting their solution to run on the SOA platform, we are still years aways from wide deployment of these next generation applications.

It for this reason, it may make sense for the packaged application vendors to actually focus on retrofitting their previous versions of applications with the SOA infrastructure such as replace proprietary work flow by BPEL or XPDL, service enable (better) some of the business processes/services. It should be done in such a manner that the developers could still use the existing (proprietary) tools, keep the current functionality as is, limit functionality changes and roll this out as a patch release for the existing ERP, CRM applications.

This approach shall accelerate the adoption of SOA within and enterprise. In addition, enterprises shall be willing to pay incremental support to get this capability, a win-win for both the vendors and their customers.

SOA real-life case studies

The SOA Consortium today published real-life SOA Case studies which are available here. These case studies are categories by types of projects (proof-of-concept, Business led projects, IT led projects and Mega projects) and industries.

The entire set is also available as a pdf, if required.

Defining Business Agility

There has and continues to be a lot of discussion around Business Agility and following is how I would define Business Agility

Business Agility = Business Alignment + IT Flexibility

Business Alignment: Alignment between the various business units
IT Flexibility: Alignment between Business and IT + ability to rapidly deploy new business capability

My observation has been that business is pretty flexible and has the capability to create a new business model effectively, if required (of course under the right leadership). The real challenges are IT's ability to integrating the new business model with the existing infrastructure. My experience has been that adopting SOA enables IT flexible to adapt to these changing business models.

Business Agility could also be defined as:

Business Agility = Business Intuition + IT * (Business Alignment + Flexibility)

Business Intuition: A common sense approach for aligning the various business unit

Business Agility = Business Intuition + IT * (BPM + SOA)

BPM: for aligning IT with the Business (yes! the business defined the business process but it is the business analyst - typically from IT captures this. Very rarely have I seen business actualy use a BPM tool for modeling business processes)
SOA: for providing flexibility

SOA Postulates, Theorems & Corollaries

Similar to the mathematics, I felt that there is a need to define the laws for Service Oriented Architecture based on facts, observations and technology roadmaps and have termed tham as SOA Postulates, SOA Theorems and SOA Corollaries.

SOA Postulates:

1. A service is an indivisible unit of work
2. A service is independent of the protocol or implementation
3. There will be one and only one service producer
4. There can be multiple instances of the same service
5. An entity that utilized the service is called the service consumer
6. There can be one or more service consumers for a given service
7. A line between the services is the agreement between the producer and the consumer
8. A service can invoke other services, thereby, creating a hierarchy of services
9. A service not consumed by any producer is an orphan service

SOA Theorems:

• SOA Theorem #1 SOA Governance observers Newton's laws of motion
• SOA Theorem #2 IT Funding observers Archimedes' principles
• SOA Theorem #3 The need to Enterprise Service Bus shall diminish over time
• SOA Theorem #4 Service hierarchy should not exceed more than three levels

SOA Corollaries:

• SOA Corollary #1 Dynamic Service Discovery does not require centralized repository (Corollary to SOA Theorem #3)
• SOA Corollary #2 All service containers need to count and limit service hops (Coroloary to SOA Theorem #4)

For those not familiar with these tersm following are the Wikipedia defintions

Postulates: The term postulate, or axiom, indicates a starting assumption from which other statements are logically derived. It does not have to be self-evident (constancy of the speed of light in a vacuum is not self-evident, however it was used as a postulate in the special theory of relativity). Some axioms are experimental facts, but some are just assumptions not based on anything.

Theorems: In mathematics, a theorem is a statement, often stated in natural language, that can be proved on the basis of explicitly stated or previously agreed assumptions. In logic, a theorem is a statement in a formal language that can be derived by applying rules and axioms from a deductive system. This definition in logic is crucial in fields such as proof theory that study the general properties of provable and unprovable statements.

Corollary: A corollary is a mathematical statement which follows easily from a previously proven statement, typically a mathematical theorem. The use of the name corollary in place of proposition or theorem is usually subjective: proposition A is a corollary of proposition B if A can be deduced quickly and easily from B, but the meaning of "quickly and easily" varies depending upon the author and context. Sometimes a corollary has a proof which explains the derivation; sometimes the derivation is considered to be self-evident.