Key Learnings - Using EDA to implement the core SOA principle of "loose-coupling"!!!

A lot has been said about how SOA and EDA are unique "architecture styles". It seems like only one or the other architectural principle is considered in proposing architectural solutions. However, there is a distinct benefit to using both paradigms in unison in solving business problems!!


Business events are the "core concept" that drive any EDA implementation. On the other hand, decoupling business applications and the business functions/ business processes embedded in these business applications is the core theme of SOA. SOA implementations rely on the use of standards based web services technology stack and that of canonical business documents (XML) .

However, it is my contention that an enterprise that does not invest in the web services or ESB technology can still leverage EDA style business events to implement loosely coupled business services provided it makes an effort to analyze its' business events and creates canonical representations of these key business events. This could mean defining business events that encapsulate a business concept that have an associated business concept state indicator or a business action indicator. Further, these business events can be used to trigger constructs such as event handlers that act as a facade or layer of indirection to execute a business function via the use of an application API.

It must be noted that the terms event producers and event consumers or publishers/ subscribers are being used loosely to denote the initiator of the event and the owner of the business behavior that "knows" how to deal with the event. In a SOA realm this would be the service consumer and the service provider respectively.

The key to this model in leveraging SOA is the use of self-describing canonical business events that are subscribed to by independent event listeners. These event listeners and/or event handlers that are delegated to by these event listeners help insulate the event producers and event consumers from the complexity of knowing how to interpret the events. Here the event producers/ publishers and event consumers/ subscribers are decoupled from one another via the use of canonical business events as well as messaging technologies.

Either of the two layers i.e. the event publisher or event subscribers can be altered long as the contract is adhered to in terms of the canonical business events. Also, messaging technology oriented configuration consoles allow the definition of the event publisher/ subscribers to be connected via metadata as opposed to hard-wiring these in code. Event handlers act as event adapters in that these could translate the event and invoke the required API call to deal with the event. The event handlers act as a business facade that hide the workings of the business application and allows the business application to change without affecting the event producers.

To recap, if loose-coupling is a core SOA principle that promotes business agility then the use of event handlers invoked using messaging technology and canonical business events can be used to deliver this goal. The enterprise does not need to invest in a SOAP stack right away to achieve this goal. If desired technology standardization and interoperability can be introduced at a later phase by turning these event handlers into web services. This two phased approach defined above enables the enterprise in pushing off investment in the technology stack to a future phase without sacrificing business agility and/or offering novel business capabilities.

 
Please feel free to drop me a note.
thanks.
surekha -

I have been busy developing globalized SaaS products

It has been a while since I last blogged, especially after joining Intuit in 08-2008. For those not familiar with Intuit engineering culture, Intuit is one of the most admired companies above the other famous companies and can attest to that (feel free to drop me line if you want more details).

OK! enough about where I work and just a quick reminder to the readers of this blog - the comments on the blog represent my own personal view and not that of my employer or any third party (with disclaimer in place can now continue :) ).

For over a year now I have been working on a personal finance tool product called Intuit Money Manager which was launched last month in India through our partner Money Control - click here to learn more about the product and click here to get started with a 90 day free trial.

So what had this got to do with Strategic Use of Information Technology? Well! for those interested in managing your own money (like me) - we have been using tools like Quicken, Money (Microsoft withdrew the product from the market last year) and Mint.com which has been very useful in helping plan our finances.

Intuit Money Manager takes only few minutes for the consumers in India to get started and with capabilities like account aggregation, auto categorization, setting goals, reviewing trends (income & spending) and tracking investments provides a comprehensive PFM tool that is configured for the Indian market. For the folks in the US - I would direct them to Mint.com which provides similar capabilities and have personally been using this product since they launched.

In the subsequent blogs I shall try and provide some additional insights on developing globalized SaaS product and how adopting Services Oriented Architecture has helped us achieve this at a much faster pace than traditional development methodology. This approach is not just adopted by our product teams inside the company. Intuit does also provide 3rd party developers to access services at the Intuit Partner Platform and would recommend you check it out, especially if you want to develop and launch a SaaS based consumer or small business product(s) rapidly.

More to follow later and as usual feel free to drop me a line with your comments/feedback.

- Yogish

Economy and IT

All the bad news about economic downturn got me thinking that there are a few parallels to be drawn between the cause and effect of current economic situation and IT. Just like American consumer, we have to start thinking about making some adjustments that will be required in future. Lets look at some of the parallels first:

Exotic instruments – One of the reasons financial system came crashing down was the invention of exotic financial instruments such as Collateralized Debt Obligation (CDO), Credit Default Swap (CDS) etc. These instruments were not well understood by majority of people that were peddling it or those that bought into it. We in IT world have been living in alphabet soup of our own – EAI, AI, SOA, BPM, CEP etc. While some of these acronyms have legitimate meaning, most are there to sell products or consulting services. In financial world these instruments have led to illusion of profit thus fat bonuses for undeserving people. In IT world this has also led to fat profits or career advancement for undeserving people

Sub-Prime Mortgages – IT equivalent of sub prime mortgages is the foolish investments we have made over time in technologies that have failed to pay back (they are the financial world's equivalent of "Upside Down" mortgages). Now, any new technology investment is risky and we should not be totally risk averse otherwise we will miss the opportunity to take advantage of transformative technologies such as the Internet, but at the same time we in IT tend to over hype any thing new. "If only I go ahead and buy a new CRM package, I will have better customer relationship." I am sure every organization has a number of expensive software products that have either been abandoned or are under utilized.

Next time I will talk about the adjustments. Anyone here with me for a “BAILOUT”?

Ashok Kumar

SOA is not.....

Looks like there is a contant need to educate the industry on SOA and this time I shall take a stab at what SOA is not....

1. SOA is not about technology
2. Web Services is not SOA
3. SOA is not dead - it has the same symptoms as global warming (too much pollution in the air)
4. SOA is not defined as "A camel is a horse desinged by a committee"
5. It is not a case of Chicken (Business Architecture) and the egg (SOA)
6. SOA is not entirely about reuse
7. SOA is not expensive - it follows Archimede's Priciples :)
8. SOA is not a product or a platform
9. SOA is not about registry & repository
10. SOA does not start with a big bang

Just my thoughts and please do feel free to drop me line with your comments and/or feedback.

Yogish

Key Learning from reclycling

Over the long weekend I decided to recycle all the boxes that were in my garage and following were my observations:

• I had not recycled the boxed for a very long time - so it was complete mess in the garage.
• About a decade back you could just put the boxed out and the garbage collectors would take it for recycling - not any more. We now need to break down the boxes - flatten them and stack them in the provided container in a systematic manner - otherwise they WILL leave them behind.
• As I was breaking down the boxed - realized that the technology has changed. It is very easy to flatten and stack them.
• However, there were a couple of boxes where I need to use brute force, cut them and even stand on them to forcefully flatten them. Some of them were recentely manufactured ones - guess they did not use the latest technology
• These days - eveything seems to come in box, no matter what you buy.
• It is important to recycle the boxed on a weekly basis - instead of waiting for a long time. The longer you wait the worse it gets.

Hmm!! - haven't I heard this before? :)

Yogish

Comparing Current Financial Crisis to SOA - Continued

I agree completely with Yogish's views as posted on "Comparing Current Financial Crisis to SOA" and his advice on treading light and having solid justification prior to undertaking SOA style projects.


A) An upfront investment has to be made in performing the right level of business process analysis and business architecture to pick the right SOA service candidates.


B) One should not attempt embarking on SOA with a highly visible project, as these types of projects operate under unwarranted pressure for delivery and with little patience for "architectural constructs" SOA or otherwise!!


C) Any project that is a high return type process with established KPIs and benchmarks is a good candidate for first time SOA implementations. Here gains are measurable (in the form of cost avoidance, cost reduction etc.) and can be directly tied to SOA-enabling these operational processes.

D) One has to invest in marketing the success of these projects and the services that were responsible for these quantifiable returns delivered by following SOA principles. Advertising the benefits of SOA style services with the benefits rendered by these projects help the business to relate to the abstract constructs such as SOA and so offer a better chance for service adoption. Also, advertising these business services that deliver efficiencies can be shown to offer speed to market gains that can be used for upcoming projects by encouraging reuse.

E) As always one should not assume technology will be the magic bullet. An assessment has to be made to understand if all components of the process/ business capability have to be refactored into services. One has to be open to leveraging legacy assets to reduce risk even if all that can be achieved is establishing a document based interaction or a mediated invocation of the legacy system.

Your feedback is invaluable.
Thank you!!
Surekha -

Comparing current financial crisis to SOA

Comparing the current financial crisis with SOA was one of the topics we briefly discussed at the Community of Practice working group of the SOA Consortium. Following are some of my thoughts on this topic:

• The current financial crisis is based on a flawed foundation - the sub-prime mortgages
• Investment bankers and mortgage companies composed new and complex derivatives and resold them all across the globe - all in the name of new and investment models
• Most of the executives of the involved companies as well as government agencies reviewed some of the aspect of the new world and based on what they knew then - thought it was all OK.
• No one seriously took time to review the potential business risk and take corrective action before it was too late
• The investments were so interdependent which made it impossible to understand the implications or the impact of letting one service fail. Example: The is no rational on why Lehman Brother was allowed to fail? and why AIG was rescued?

...and we see the result today. Doesn't this sound familiar?

Following are my recommendation on how to make sure that our SOA adoption does not take the same path (some will).

• Make sure that all SOA adoption is based on sound business and technical foundation. Adopt and make sure to document the business architecture (design) and develop a technology road map (architecture) to meet these goals.
• Do not hype and push rapid adoption without thinking through the life cycle .
• Ensure that there is proper governance around your SOA Adoption - over communicate and be a bit more conservative on the risks.
• It is not necessary to adopt SOA for all implementations - I would rather then to recommend that one makes sure that some of critical business capabilities (applications) are provided (build) using the traditional model.
• Do not depend on services that are more than two level deeper (Reference: SOA Theorem #4: Service Hierarchy should not exceed more than three levels ). This is to ensure that you understand each of the services and their performance which will help you take corrective action, if required.

These are just some of my initial thoughts and as usual please do feel free to drop me line with your comments and/or feedback.

- Yogish

Canonical Models and Services

In thinking about canonical models one mostly thinks about an enterprise worthy or industry compliant representation of a business concept or a business entity। Often this canonical model compliant payload is packaged in a response envelope that is returned by the service provider. However, the question is whether the term canonical model can be used to define the " business request" or if it is limited to the "business response". This blog explores the canonical request models that might be used to alter the behavior of enterprise services using search services as an example.

Let us look at a "search service" where one could think of a "context based search request" that is made to the search facility to make the search behavior of the service providers more efficient। In addition, this "search context" could also be used to express the specific search needs of the service consumer. Given this, the question is whether the canonical model could be used to define a request model that is a representation of the service provider search/ browsing parameters and service consumer's usage context. Here the search capabilities of a search service could use search/ browse related grammer rules to interpret the search context that is embedded into the request by the consumer. The context provided by the consumer allows the service provider to "accurately" and "efficiently" interpret the request.

For example, search semantics embedded in a search service that returns cross-sell product options could include information like "customer preferences", "type of credit card used by customer", "shipment processing preferences", "customer purchase history" etc। This type of "customer information" represents usage semantics embedded in the request and allows the provider to return cross-sell product options that are geared to the the specific customer being refered to by the consumer call। This in turn allows compilation of product suggestions which incorporate the profitability levels and purchase ability of this particular customer. The "customer information" is deemed to be the search result and usage context that are sent into the search request. These search results are thus customized to the consumer context without the consumer needing to make multiple calls to get the desired results. Therefore, one could see how the canonical request model has not only the standard search parameters supported by the provider but also allows the consumer to express its' search context and how it might apply the results of the search/browse call.

In addition, the canonical request model based search context could be used by the consumer to drive processing efficiencies in the provider। These request based keywords/ context can help the provider eliminate or short-circuit certain types of processing as well. For example, a canonical request model could support "new customer search" vs "existing customer inquiry" keywords to help alter the providers' audit processing behavior and past inquiry Here a consumer call with "new customer search" context could be used to suggest to the provider that the audit/ security and past inquiry lookup be disabled thus positively impacting the response time of search call.

In conclusion, we can see that the consumer usage semantics allow the provider to tailor it's search behavior to the needs of the consumer without changing the provider service interface.

Please give me your feedback on this topic। Also, I would be interested in knowing whether or not you have leveraged these concepts in your industry vertical।

Surekha -
Thank you!!

Organization Skills assessment and Reference Architecture

The other day I was participating in the engineering skills assessment discussion and it dawned on me that mapping the skills to a reference architecture makes is much easier and simpler discussion to have. The reference architecture enabled us to first list the categories and drill into each of them to define the skills as well as leveling required for Architects, Developers, QA and Managers positions.

Key Learning's:

• Having a reference architecture also helps in organization (engineering) skills assessment
• The Skills Assessment / Mapping is independent of Governance and Organization. However, Governance drive where these resources belong
  

SOA Consortium announces SOA Case Study Winners

The SOA Consortium and the CIO Magazine this week announced the winners of the SOA Case Study competition. Please click here for the results.

Best Practices: Master Data Management

Following are some of the best practices for adopting (note - not implementing) Master Data Management solutions within an Enterprise.

Understand the Business Context (semantics) prior to picking a solution

As per my earlier blog on EA, BPM, SOA and MDM it is very important to understand the business context, including the semantics of what each of business units, departments and entities mean when they refer to MDM entities such as Customers or Products. For example marketing deals with leads, sales with opportunities/accounts and services with paid customers. Should all the entity be referred to as customers throughout the business process? or should the master entity be referred to as Organization? Common vocabulary goes hand-in-hand with master data.For example what does this sign on a building mean? Is the building fully occupied and available for purchase? or does it mean that the entire building is unoccupied and available for rent? The business context needs to be clearly defined in terms of business units, departments and business processes.

Develop a comprehensive entity model.

Not only should one define the common attributes for the master data, one should map the entire set of attributes and where it is used. For example, in customer services the customer is used as a reference whereas in order management it is transactional data.
A set of models and documents that can be used both by Business and IT.

Establish data governance process right up front.

It is important to establish data governance process right upfront, especially as this may required dedicated resources from all the business units as well as from IT to manage the master data. The approach that worked for us is as follows:

• Executive Leadership Team Data Leadership Team that meets on a periodic basis (once a month) to establish data policies, standards, establish priorities for the data quality team
• Data Stewardship (Quality) Team are the business operations people who manage the data quality on a day to day basis. The business team enforces and ensures data quality across the enterprise.
• Enterprise Data Program team is responsible for developing and managing the programs and business rules in the various technology tools.

Integrate with 3rd party data providers

It is important for enterprises to consider bringing in 3rd party data providers such as D&B and Factiva for better understanding of the market. For example on a typical morning between 9:00am and 11:00am

• 706 businesses will move
• 578 businesses will change their phone numbers
• 60 businesses will change their name
• 49 businesses will shut down
• 10 businesses will file bankruptcy
• 1 business with change ownership

Source: Dun & Bradstreet (2004)

3rd party data from D&B could be used for leveraging legal name as the name of the organization, providing knowledge management systems that also provide news feeds as well as market statistics by industry, geography and demography and also mapping it to existing sales.Picking the right data standardization and matching engine:

This is a key technology that will either make or break the quality of your data. I agree that developing the business rules and configuring the matching tool is more important - however from a technology point of view - I would dedicate substantial amount of time evaluating and testing the tools with existing data before picking a technology. My preference would be to use one of the following matching engines:

Trillium, First Logic (now SAP), IBM or SAS.

In cases where a MDM product is packed with a different matching engine, I would be tempted to externalize the data quality to one of the above mentioned data matching engines. Just my preference - maybe the other quality engines may have got better. Do you own evaluation.

Expose all MDM functionality as services
Expose all functionality such as data (address) standardization, data matching, update master and propagate master key.

As usual please do feel free to drop me line with your comments and/or feedback.

- Yogish

Enterprise Architecture, BPM, SOA and Master Data Management (MDM)

One of the best practices for Enterprise Architecture teams to redo the enterprise road map on a periodic basis. It is typically reviewed and updated during the yearly budgeting cycle and my preference is to perform this activity every 18 months. The best practices (and the traditional approach) is to first document the as-is, next develop the target or future state (architecture) and finally develop a short term (6 months), mid term (12 months) and long term (18 months) road map. Preferable an actionable road map that ties back to the business initiatives.

It is good to document the the as-is (or current reality) from all the domains such as Business Context, Applications, Technology, organization and Funding. Typically the business context is best understood by identifying and mapping the key business processes at a high-level.


This approach not only helps have a common vocabulary between business and IT by identifying the key business processes, it also helps identify the key enterprise data objects (entities) such as Customers, Contacts, Products and orders. Based on the priorities of the each of the business process, the next steps would be to drill down into one or all the business processes as illustrated below.
Once again, it is not necessary to use a Business Process Modeling tool (however, using one would be helpful later), the objectives is to clearly identify and document the next level of details. The next steps are to perform the gap analysis on each of the activities as illustrated below.

This approach enables both business and IT clearly visualize the existing gaps, impact areas and cost estimates which helps in developing the priorities and the investment plan. In addition, it is also important to identify and illustrate the list of applications/solutions that support a given business process as well as it perception within Business and IT as illustrated below.
As we develop the actionable road map to the future state, one this is obvious, no matter what we implement or adopt, whether it is a packaged applications, BPM or SOA key enterprise data crosses the silos both from an organization and the applications. It is for this reason, there is a critical need for adoption Master Data Management across the enterprises.

It is very important to spend sometime understanding and mapping these enterprise data objects in the context of the business process. It would also be very helpful to also develop a high-level data model and transaction (CRUD) matrix associated with the business process before initiating the activity of selecting/implementing an MDM solution.

I have seen a lot examples where companies have embarked on an MDM project without developing the architecture (see my blog on Blueprinting Information Architecture for more details) and not meeting the desired business outcome. The two other primary reasons of MDM failures are:

• Lack of developing the data governance model up front that involves all the impacted business units
  
• Assuming that a packaged applications could be modified to be the master data for enterprise.
  

To keep this blog short, I plan to blog more about my thought on BPM & SOA later and please do feel free to review my CDI-MDM Summit presentation on my experience with MDM/CDI.

As usual please do feel free to drop me line with your comments and/or feedback.

- Yogish

Key learning from Home Entertainment/Automation that can be applied to SOA/SaaS

Unlike the previous generation where technology innovation was driven by enterprise needs, over the past few years technology innovation such as smart phones, multimedia, game consoles, multimedia and social networking is being driven by the consumers. In short the consumers have gone digital, whether it is HDTV, Blue-ray, Home automation, smart phones, media servers or IMS (IP Multimedia Services). The vendors manufacturing these devices and services understood that unless there are standards adopted by industry - the consumers would not adopt these technologies (especially as they are not cheap). It is for this reason, most of the large vendors (hardware, software, manufacturers, protocol stack providers, etc.) got together to form the Digital Living Network Alliance (DLNA).


Their objective was to resolve the following consumer challenges:

• Products designed for the home should be easy to install, provide noticeable user value and be affordable
• Product must inter operate with each other without requiring the consumer to undergo complex setup and configuration for connection between devices
• Digital home products must inter operate with each other and with existing CE devices such as TVs and stereos.

Doesn't this sound very similar to the current IT Operations challenges?

The above diagram illustrates DLNA's view of the customers needs (source: DLNA). In order to be vendor neutral and provide the consumer the ability to control any service (yes! they call it services) the DLNA members standardized the technology stack (as shown below - source DLNA).

The key learning here is that the vendors adopted Peer-to-Peer (P2P) for device discovery, control and Media management. For now they have all adopted UPnP, especially as most existing devices at home (desktops, laptops, storage devices, game consoles, IP based TVs, Stereo systems, network hubs, etc.) support UPnP. Some of the vendors such as Microsoft support both UPnP and WS-Discovery and it the long term (once the backward compatibility issues are addressed) the industry may migrate to WS-Discovery.

For those getting ready to purchase TVs, Phones or other Digital Devices, I would recommend you verify that they are DLNA Certified.

The next obvious question is What does this have to do with SOA/SaaS? Well! why not use this same approach for deploying services? It would make life much easier for IT Operations and potentially eliminate the need for additional ESB hops in the network. Yes! I am back on this topic :). The Services-Oriented approach, it is basically a P2P architecture, i.e. a consumer invokes a producer. As most of the large software vendors have made a commitment to adopt Services Component Architecture (SCA) and developing the SCA Run time engines, it would great if they could adopt either UPnP, WS-Discovery or some other P2P technology. The following diagram illustrated the joining of new node/service(s) to the P2P network.

Benefits:
Following are the benefits of adopting this approach:

• Based on the SCA standards -unique (logical) service name for services both for defining and invocation. Do not need to know the EPR (physical location) at the time of deployment.
  
• Multi-cast availability of service whenever an instance come up - dynamic configuration does not require IT Operations or tools (even if they are automated) to change configurations.
• Service maps (for a predetermined domain/network) maintained at each node. Complete map could be maintained in the Super peer (read up on p2p architecture for more details).
• Eliminate the need for Service Registry in production. As each instance of the node and services is maintained dynamically by the Super Peer - there isn't a need to maintain and administer a Service registry.
• Eliminates the need for having a separate monitoring agent on each node, especially as each instances could updates it's own service performance details in the P2P map.
• Universal administration tool could be used to configure one or all the instances at any node and propagate the changes across the network.
• As the consuming services would know the EPR of the producing service, this eliminate the need of an ESB.
  

The Newton: Component Model (Key technologies, OSGi, Jini, SCA) is the only run time engine I know of, that supports both P2P (Jini) and SCA. The Apache Tuscany project does claim to support JXTA (P2P) binding and have not researched it as yet.

Just my thoughts and as always do feel free to drop me line with your comments and/or feedback.

- Yogish

Mashup, WOA, SOA, SaaS, REST and the kitchen sink

These days there is a lot of jargon thrown around and following is my attempt to make sense of put these all together in a simple terms. A could of year back some of us, the early adopter of SOA, had put together the Enterprise SOA Maturity Model (Presentation). The maturity model for IT organizations (in simple terms) consisted of three levels starting with initially developing web applications, followed by composite (aggregated) applications and finally maturing to end-to-end (automated) business processes.

It is great to observe that some in the community are very strong advocates for Web-Oriented Architecture(WOA) as a first step towards adopting SOA. This basically validated our initial assumption that business will only fund those projects that demonstrate value and providing web based solutions is the easiest way to get their buy-in. For those not familiar with WOA - you may want to read this Blog on What is WOA - The Future of Services-Oriented Architecture . What WOA does is simplify the architecture and approach - however there is still a need to ensure consistent architecture and governance across the enterprise.

The above diagram illustrated the architecture approach for WOA based sample application. To keep it simple - this sample applications integrated with Identify Management, Stock Information and a Business Process. The RESTful approach makes is simpler and easier. However, the deployment model can very quickly become pretty complex, especially with the introduction of ESB as a mediation layer. I completely agree that this is how I deployed and may still adopt the model again in the near future. However, as the number of services increases, it increases the burden on the Architects as well as IT Operations to manage not only the complexity of services dependencies, but also the complexity of multiple instances of the same services in the production environment. Coupled that with the introduction of ESB as a mediation layer and the configuration of the load balancer and firewalls between the various sub-zones in the network - this model could become unmanageable (even with an SOA Repository + CMDB) very quickly.

Alternate Approach:
Last year while working with my colleges we came across Tuple Space (an implementation of the associative memory paradigm for parallel/distributed computing) a theoretical underpinning of the Linda language developed by David Gelernter and Nicholas Carriero at Yale University. The original Linda model requires four operations that individual workers perform on the tuples and the tuplespace:

• in atomically reads and removes—consumes—a tuple from tuplespace
• rd non-destructively reads a tuplespace
• out produces a tuple, writing it into tuplespace
• eval creates new processes to evaluate tuples, writing the result into tuplespace

This simple approach of handling Tuples (Objects) in memory should enable us to integrate the enterprise using simple (limited set of) operations, similar to what the RESTful style.




The above diagram illustrated a simpler approach of integrating the enterprise (or enabling composite applications). From a business context - they always deal with an entity such as Customer, Order, User, Inventory and Product. This approach enables the IT Project teams to understand and define the business context in their language .


How does this work?
There are various different technologies and for this example - lets take JavaSpaces. Every objects has it's own unique space. The number of operations that can performed on a space are limited to Write() - Create/Update the space, Read() - Read the space without delete, take() - read and delete the space and Notify() - send an event when ever a space changes (write() or take()). Of course, they had additional commands and some vendors have provided SQL interface to these objects.

Impact to IT Operations
As organizations move to the objects (space) model - it makes it easier for IT Operations to manage multiple instances of the distributed (in-memory) data grid, eliminated the need to know and understand the services dependency matrix. This approach does not eliminate the need for ESB but does limit the need of multiple (logical) hops and reduces the number of (proxy) services that need to be maintained and managed by IT (development and operations).

Impact to BPM
Business is very much interested in knowing, preferably in real-time. the status of their business. Unfortunately, the currently BPM products can provide BAM capabilities only for those business processes that run on their instance. None of the existing products have the capacity to provide end-to-end near real-time BAM capability.

This approach enables IT to provide business monitoring and management functionality by leveraging the notify() capability of JavaSpaces.

In my opinion, in the long term the BPM vendors shall be providing the modeling and simulation capability and the Business Process Execution shall be powered by distributed (in-memory) object infrastructure.


Impact to SaaS
Today most of the SaaS Platforms (also referred to as PaaS) are object driven. The objects and their relationship not only drive the user interface but also the Business Process and Rules management. They are already one step ahead of the rest of the industry. However, most of SaaS infrastructure and platform teams spend a lot of time an effort in dealing with multi tenancy.

The Distributed object technologies provide capability to map the objects (spaces) to data sources (Databases, JCAs, WS, Files, etc.) one could potentially implement a multenant SaaS solution powered by an existing packaged application at the back end. Something that the existing SaaS companies should not ignore.

Business State Machines
As discussed previously - business deals in the context of entities (or business objects) and their decision making criteria is based on something happening (events). A combination of Event Servers and Business State Machines may be sufficient and enough to provide an end-to-end business process management (run-time).

Impact to Master Data Management (MDM)
In my opinion, MDM is a must-have solution for any enterprise that is interested in managing and monitoring their end-to-end business processes, independent of whether they adopt SOA or not. As MDM solutions deal with the most common business entities (objects) such as customers, contacts, orders and products the distribute (space) object model fits in very well. Address Standardization, Matching, reading entire customer record and performing real-time aggregation (analytics) becomes much easier to develop and manager. An approach that existing MDM vendors should look into. Most of the MDM solutions provided by vendors are easier to implement (compared to 5 years back where we had to build it all within IT), the biggest challenge with these solutions is defining and maintain the Master Data in business context.


Summary:
Even though the traditional approach scales to meet large enterprise needs, it is too complex and difficult to provide for true agility. The distributed space concept has the ability to facility business agility through plug able modules (based on event/state change notifications) and map the objects back to a source system. This not only changes the game but also takes the industry one step closer to the reality of cloud computing.

Just my thoughts and as always do feel free to drop me line with your comments and/or feedback.

- Yogish

IT Engagement Model

One of the key ingredient for success is clearly defining the roles and responsibilities within IT. There are multiple stake holders in IT with each doing their best to provide the highest level of support to the business. Most of the time this results in people stepping over each other - especially as there generally is a not a clear definition of everyone's task. Most of the project failures are due to the confusion is the definition of the roles between the PMO, Project Manager and the Enterprise Architects, resulting in responsibilities overlap and lack of making decisions.

It is important to do this in the context of the Services Life Cycle and did publish a short presentation on this topic a couple of months back. This presentations is a summary of the SOA Practitioners Guide Part 3: Introduction to Services Life Cycle with the addition of the IT Engagement Model slide (14) shown below.





This is the RACI (Responsible, Accountable, Consulted and Informed) slide listing all the roles and responsiblities of various actors across the services lifecycle. This is pretty generic in nature and could also be applied to non-SOA lifecycles too.

You comments are always welcome and do feel free to drop me line.

- Yogish Pai

Explaining the terms "Services-Oriented Architecture"

Recently there has been some interesting discussions on multiple on-line groups on defining Services-Oriented Architecture. My observations has been that a majority of the participants on the on-line forums refer to SOA as an Architecture Style. I would agree with this, however I would not mention this to the non-IT staff. What would they care or know about (Enterprise) Architecture Styles? and does it make any sense for us to explain this to them? We might be better off learning more about their needs instead.

Wikipedia definition starts with "SOA is a software architecture...." and because of this, I would not agree with it. Maybe one of these days I shall get enough courage to change it on Wikipedia :) .

I just read Surekha's blog on Key Best Practices: What is Serivces-Orientation? and am in complete agreement with her that

There is no place for the use of architecture terms, technological jargon or even as much as a mention of the web services or the WS-* stack in this definition!!

I would still define SOA as described in the SOA Practitioners Guide Part 1: Why Services-Oriented Architecture? . The SOA Practitioners defined it as follows:

SOA is a business operations strategy for leveraging information to meet the organization’s objectives, such as increasing overall revenue, boosting customer satisfaction, improving product quality, and enhancing operational agility.

Following are the reasons why this still makes sense to me:

First: Business is responsible for their own strategy (some interesting anecdotes) and yes! IT should (note: not MUST - IMO) have a seat at the table to help influence the strategy. Once the business stategy is established it is the responsibility of Business Operations to execute on the strategy. This is where IT comes in - parnering closely with Business Operations in developing the Business Architecture. People from the IT Organizations generally interact with Business Operations on a day-to-day basis. The IT Leadership does interact with the business executives but more on a periodic basis. It is for this reason I prefer the term SOA is a Business Operations strategy for leveraging information.

Next: I like all strategies to be actionable, i.e. a roadmap for achieving specific desired goals. For example, the primary goal for SalesForce.com (Disclaimer: I know as much about SalesFoce as most of the readers of this blog) is to most probably to become the primary platform for ALL SaaS solutions in the market. In this case, the business operations folks (sales, services, marketing, finance and IT) have most probably defined approaches for meeting this objective (which could be in form of marketing campaigns, events, sales incentives, partner programs, etc.). In order to execute this strategy they will require services from their own internal IT (yes! they do have an IT Organization) to provide them the capability to execute and monitor the progress (...leveraging information to meet their objectives).

Finally: I agree Todd's and Rob's thoughts about the term "The Business" - my observation is that this is not limited to IT Organizations. This is true for all organizations where I have heard sales executives refer to the "Those Marketing Operations folks" and so on. The only company where I had seen systematic approach by executives to rediret the entire company to consider themselves as a single team was at The Coca-Cola Company (or at least that was my observation in the early 90's - when I was in their IT organization). So even though I agree with their frustration with the term "the business" - I still prefer the term Business Operations in the definition.

Hope this was interesting and do drop me line to discuss this further, if interested.

- Yogish Pai

Intel enables SOA without ESB

In one of my previous blog on A Decision Makers concern about ESB I had mentioned that overtime the need of an Enterprise Service Bus will diminish over time. Annie Shum just send me an email with a link this blog on New Offerings from Intel Enables SOA without ESB - guess we are on our way.

Vendor need to adopt "Common Sense" Strategy

A lot of vendors are jumping on the band wagon about the latest jargon such as SOA, Web 2.0 (or Enterprise 2.o), BPM and now WOA(?). It does make sense from a marketing point of view, however they also need to adopt a "common sense" approach.

Following are a few of suggestions to the large ISVs (Independent Software Vendors).

The Platform is the application:
This is true and that a majority of IT organizations are demanding, starting right from the CIO down. This is true not only for enterprise solution but also for consumer solution, resulting in all major Internet (what do we call these companies now? ) companies such as Google, Yahoo!, Facebook, eBay and Amazon.com are taking this approach. For this discussion I shall limit it to the large Enterprise ISVs.

All the major ISVs have announced and invested heavily in developing their next generation application based on a unified middle ware. The message I am hearing from these vendors is that they will support the current version for quite a while (so if you have PeopleSoft 8.x you may be fine until Oracle decides to pull the plug) but they do expect you to upgrade (migrate - and yes! this cost you a bundle!!!!) to the next generation.

This approach really puts the IT organization (customers) in a spot, more so from people skills point of view. First, it will take time for the current staff (or SI partners) to come-up to speed with the new tools sets and second, the early adopter may pay the price to finding major bugs in the field, especially for their mission critical applications such as Finance, Order Management, Supply change, etc. Don't want to have major problems there. I would like to once again repeat that vendors need to incorporate SOA infrastructure in legacy application. If this does not work for them, they why not support the existing (legacy) applications from the next generation development ?

Lets take the case of an IT organization using PeopleSoft 8.x. Why not provide the PeopleSoft developer to developer to develop/modify existing work flow/application logic using their SOA Platform development tool? Agreed Oracle will have to invest in making this backward compatible for deploying the solution to PeopleSoft 8.x. However, this approach will get the existing support engineers up to speed on the latest tool resulting in faster adoption of the next generation platform.

End-to-End Life cycle Management:
Most ISVs, especially the large ones, do not provide an end-to-end life cycle management for their product stack(s). Of course, they will definitely claim that they do - but not really. How many software companies provide the capability for CIO and the IT leadership team keep track of all their major investments? also, how many of them provide the capability to get realistic feedback of the solutions deployed? None - as far as I know. Most of the Strategic IT organizations have been able to put this in place by procuring multiple solutions (IT Governance tools, EA tools, BPM, Enterprise Monitoring and Management Solutions, BI, etc.) and a lot of $$$ to stitch them together. Wouldn't it be great if the ISVs put something together that could be plugged in? Even if it just only for their solutions?

Provide Complete Solutions:
ISVs must provide complete solutions - not just point solutions that requires a lot of time and resources to customerise the solution. Lets take the case of Master Data Management (MDM) - all major vendors provide this solution (and one of them has 4 MDM products, including UCM) but is only a point solution. Why don't the ISVs integrate their MDM solutions with at least all their major (if not all) applications such as CRM, Order Management, Finance, Supply chain and Customer Support? Also, why not also provide the customer with a list of best practices and governance models (for free)?

It may may help if we all collectively push back on the large ISV and also have their Products Managers from ISVs spend 3 months in an IT Organization. Just a thought.

Enterprise Service Bus vs. Service Component Architecture

My attempt is to present a simplistic view of my take on the purpose of an ESB (enterprise service bus) and SCA (service component architecture). This entry is in response to my esteemed colleague's commentary on "A decision makers concern about ESB " where in the two concepts of ESB and SCA were being compared. I must add that I am in complete agreement with Yogish on the fact that ESB vendors are adding to the confusion of what is and what is not part of the "core feature set" of an ESB.

My own opinion (misplaced it might be!!) is that ESB vendors tend to bundle a lot of "service creation"utilities and "adaptors" into the base ESB stack, in an attempt to achieve "product differentiation". This very act causes ESBs to become both ridden with "service behavior" which may also lead to "slow performance" of the ESB based mediation. The reason being that it becomes very difficult to performance tune an architecture component that is doing multiple things - it breaks the rule of "high cohesion" in doing "service mediation" and "service behavior execution"!!! The former is a high through-put pass through activity while the later could have fairly intense computation business logic or data processing logic.

Now back to the ESB vs. SCA topic.

ESB - promotes the abstraction of the service provider from the service consumer. Here it is assumed that the goal of exposing the service provider interface is to enable the reuse of the service provider business behavior. ESB' core theme being service mediation.

SCA - promotes the abstraction of wiring in of the external resource dependancies and external transport protocol bindings from that of the core and reusable business behavior of the service. Here the assumption is that the component parts of service provider can be re-assembled in various combinations to assemble services that are invoked from multiple consumer contexts. SCA' core theme being service assembly.

ESB enables service interaction/ mediation from multiple service consumer contexts while SCA enables service provider component parts to be assemble for "servicing" multiple service consumer contexts.

In both cases, the attempt is to decouple service usage from reusable service behavior.

Your comments on this topic are invaluable.
surekha -

Enabling Strategic IT through Services-Oriented Architecture

Lately there have been a flood of negative comments about Services-Oriented Architecture, especially about providing business value. In this blog I have attempted to provide a simple example on how SOA, if done right, could provide substantial value to the business for an enterprise.

Read more about this at my IT toolbox blog here.