Key Learnings - Using EDA to implement the core SOA principle of "loose-coupling"!!!

A lot has been said about how SOA and EDA are unique "architecture styles". It seems like only one or the other architectural principle is considered in proposing architectural solutions. However, there is a distinct benefit to using both paradigms in unison in solving business problems!!


Business events are the "core concept" that drive any EDA implementation. On the other hand, decoupling business applications and the business functions/ business processes embedded in these business applications is the core theme of SOA. SOA implementations rely on the use of standards based web services technology stack and that of canonical business documents (XML) .

However, it is my contention that an enterprise that does not invest in the web services or ESB technology can still leverage EDA style business events to implement loosely coupled business services provided it makes an effort to analyze its' business events and creates canonical representations of these key business events. This could mean defining business events that encapsulate a business concept that have an associated business concept state indicator or a business action indicator. Further, these business events can be used to trigger constructs such as event handlers that act as a facade or layer of indirection to execute a business function via the use of an application API.

It must be noted that the terms event producers and event consumers or publishers/ subscribers are being used loosely to denote the initiator of the event and the owner of the business behavior that "knows" how to deal with the event. In a SOA realm this would be the service consumer and the service provider respectively.

The key to this model in leveraging SOA is the use of self-describing canonical business events that are subscribed to by independent event listeners. These event listeners and/or event handlers that are delegated to by these event listeners help insulate the event producers and event consumers from the complexity of knowing how to interpret the events. Here the event producers/ publishers and event consumers/ subscribers are decoupled from one another via the use of canonical business events as well as messaging technologies.

Either of the two layers i.e. the event publisher or event subscribers can be altered long as the contract is adhered to in terms of the canonical business events. Also, messaging technology oriented configuration consoles allow the definition of the event publisher/ subscribers to be connected via metadata as opposed to hard-wiring these in code. Event handlers act as event adapters in that these could translate the event and invoke the required API call to deal with the event. The event handlers act as a business facade that hide the workings of the business application and allows the business application to change without affecting the event producers.

To recap, if loose-coupling is a core SOA principle that promotes business agility then the use of event handlers invoked using messaging technology and canonical business events can be used to deliver this goal. The enterprise does not need to invest in a SOAP stack right away to achieve this goal. If desired technology standardization and interoperability can be introduced at a later phase by turning these event handlers into web services. This two phased approach defined above enables the enterprise in pushing off investment in the technology stack to a future phase without sacrificing business agility and/or offering novel business capabilities.

 
Please feel free to drop me a note.
thanks.
surekha -

Key Learnings:Drawing parallels between Design Patterns and the principles of SOA - Part II

This blog entry attempts to expand on the concepts explored in a prior blog of mine Key Learnings: Drawing parallels between Design Patterns and the principles of SOA that deals the relevance of design patterns in the world of SOA and services. Patterns explored previously were the Facade, Abstract Factory, Builder, Factory and Bridge.

In this blog we look at how infrastructure components like the ESB that are part of the service mediation layer insulate the service consumer from the service provider by offering call-dispatch functions that map out the most efficient call execution path for honoring a consumer business request। Many of the constructs of the service mediation layer provide add-on capabilities which are in fact model driven implementations of common design patterns।


Adapter - modifies an incoming method call to fit the required method signature or definition of the provider without impacting the consumer

Transformer - adapts the parameter and return type or message format (which includes alteration or interpretation of the message content) and enables the insulation of the consumer information from that of the providers'

Decorator - augments the behavior or is a facility to add on to the behavior without altering the interface and breaking the service interface and service contract as new consumers need additions to the base business behavior

Interceptor - provides ancillary behavior, filters out information or calls to the provider, validates the message content for authN credentials, authZ/ entitlements and permissions etc। all of which provide important ancillary behavior to the provider without having the providers' service implementation layer having to be peppered with nonbusiness logic level code।

Finally, the point of this blog is to show that despite vast numbers of changes in technologies and technology based service offerings the basic design principles and design patterns of the object oriented design realm are still applicable। Knowing these and applying these carefully enables one to retain the durability of the service interface and protects the consumers from unexpected or non-deterministic results and exceptions.

Thank you for listening. Your input is invaluable!

- surekha

Comparing Current Financial Crisis to SOA - Continued

I agree completely with Yogish's views as posted on "Comparing Current Financial Crisis to SOA" and his advice on treading light and having solid justification prior to undertaking SOA style projects.


A) An upfront investment has to be made in performing the right level of business process analysis and business architecture to pick the right SOA service candidates.


B) One should not attempt embarking on SOA with a highly visible project, as these types of projects operate under unwarranted pressure for delivery and with little patience for "architectural constructs" SOA or otherwise!!


C) Any project that is a high return type process with established KPIs and benchmarks is a good candidate for first time SOA implementations. Here gains are measurable (in the form of cost avoidance, cost reduction etc.) and can be directly tied to SOA-enabling these operational processes.

D) One has to invest in marketing the success of these projects and the services that were responsible for these quantifiable returns delivered by following SOA principles. Advertising the benefits of SOA style services with the benefits rendered by these projects help the business to relate to the abstract constructs such as SOA and so offer a better chance for service adoption. Also, advertising these business services that deliver efficiencies can be shown to offer speed to market gains that can be used for upcoming projects by encouraging reuse.

E) As always one should not assume technology will be the magic bullet. An assessment has to be made to understand if all components of the process/ business capability have to be refactored into services. One has to be open to leveraging legacy assets to reduce risk even if all that can be achieved is establishing a document based interaction or a mediated invocation of the legacy system.

Your feedback is invaluable.
Thank you!!
Surekha -

Architecture tenets of High Cohesion and Loose Coupling

Architecture tenets of High Cohesion and Loose Coupling – Both of these tenets are related to one construct i.e. that of a “Contract”.

The term contract in information technology involves the definition of high level interfaces in the form of a coarse-grained set of operations that have well known inputs, output, clear exceptions or faults. The contract hides all of the details of implementation and allows these hidden implementation details to behave as one cohesive unit - in that it provides support for "high cohesion". By extension, in separating the client or consumer or caller of the contract from the implementation details it provides support for “loose coupling”.

This concept of contract works at any of the following levels:
1. sub-system interface (for example, a persistence sub-system)
2. component interface (for example, a remote monitor)
3. layers of architecture (for example, business layer vs. presentation layer)
4. infrastructure service (for example, a messaging service)
5. SOA style business service (an customer account self-service)

Furthermore, the concept works whether the implementation is a local call or a remote component call long as the "contract" is honored. Experienced architects also insist on unidirectional contract-based communication even between the layers of the architecture - with communication only being allowed to the very next layer down. The concept is that the more volatile layers interact with the more stable layers’ contract without skipping levels. This level of indirection adds as a check for the entire system as the volatility of top level layers and the communications from the volatile layers’ are limited to the very adjacent layer alone without affecting multiple aspects of the system when these layers change.

This concept is the key driver of the Model View Controller or MVC pattern wherein the presentation layer is allowed to talk to the interface or contract of the controller layer or the façade layer alone but not to the interface in the business logic layer or the data logic layer. Also, the façade layer or the controller layer is not allowed to communicate to the client or the presentation layer. The contract dictates that it is the client or the presentation layer that is responsible for initiating the communication.

Advantages of adhering to the Contract:
A) Implementation details can change without negatively affecting the consumer or the client. Loose coupling facilitated by the contract protects the client or consumer. Also, since the behavior is highly cohesive (all hidden behind the contract in one well-knit codebase) any alterations to business rules or behavioral logic is embedded in this codebase, thus insuring the completeness of apply the rule/ logic change. Without this it is very possible that part of the logic is embedded in consumer codebase and part of it may also be placed in the communication or the mediation layer (leading to the anti-pattern low cohesion and high coupling).
B) System integration testing and performance optimizations are easier when there is a known finite set of operations, inputs and outputs that will be allowed by the contract.
C) Understanding the interactions and invocations for the consumer becomes easier due to the known and pre-configured set of operations that are published on the contract. This helps to make the system or application more deterministic.
D) As long as the contract is not broken in making behavioral enhancements new consumers can be entertained without having to create newer versions of the codebase. Of course, this can also mean addition of newer audit and tracking capabilities in compliance with internal or regulatory policies without affecting or "informing" the consumer.
E) Since the consumer is not interacting with multiple internal points of the codebase, the system interactions and resource utilizations per consumer/ client call are more quantifiable and predictable. This makes it easy to scale the system for availability. This contract or interface then becomes the single point of entry for all interactions and is thus the only point that needs to be monitored to assess system resource utilizations. In addition, provisioning of system resources becomes more scientific as all calls of a certain operation take a known amount of time and resources given that the inputs are also quantifiable.

As can be seen, a simple construct such as “Design by Contract” when taken seriously and to its logical conclusion renders a great deal of architectural stability, robustness and extensibility.

As always your comments are welcome.
surekha -

Key Best Practices - REST Assured????

The purpose of the blog is to find out if there is a place for REST in the realm of “Business Services”.

First of my definition for REST – it is a "Get", "Put", "Post", "Delete" operations performed on "Resources" that are identified as URIs being transmitted over HTTP(S) as REST.

REST by nature has a very simple service operation set with the complexity all embedded in the Resource URI. The operations that REST allows are NOT business user-friendly and hence do not really belong on the Service Repository that an end user is referencing to discover business services. To accommodate complex business behavior and to compensate for the finite list of RESTful operations the resource identifiers have to be fairly complex.

SOA on the other hand, allows fairly diverse interface definition that resembles as closely to the business syntax as it can get. The canonical model that is shared is standardized and is not as diverse as are the resource URIs of the REST architecture. The canonical models for the most part are XML.

Another key difference lies in the protocol and messaging support for REST. REST is limited to only HTTP/ HTTPS which by nature is stateless and has no standardized protocol level interception model for the more enterprise category behavior such as guaranteed delivery of messages and asynchronous messaging semantics. SOAP and the standardized metadata associated with SOAP envelope on the other hand can be interpreted by the Web Services SOAP stack to enable behaviors such as Reliable Messaging, Transaction Management, Addressing, and Notification with a level of interoperability across-vendor platforms.

Having said that, it is possible to find a home for RESTful architecture and to leverage one of its strong points in terms of a simplified and standardized interface. REST may be a good architecture model or construct to follow in the service implementation layer or at the service adapter layer. This helps to keep the service implementation the same across any type of service interface definition. In the event there is a need for additional transactional integrity then these behaviors could be taken care of in the service facade or in the service mediation layer while keeping the actual implementation in the simplified RESTful architectural format.

If we are looking at SOA service operation being exposed to the world and using a RESTful service implementation layer then we would need to deal with translating canonical models and parameters of SOA style service into URIs. This translation behavior includes extracting the request parameters from the Request canonical model and transforming these to verbs and values in the format of a query string i.e. a URI.

One option is to have the specific URIs needed by the service implementation layer to be translated to the input URIs as specified by the RESTful service implementation layer. Here the service interface layer or the service mediation layer performs the translation of the input parameter into the specific URI and initiates the invocation of the appropriate service implementation layer.

The other option is to have the service interface layer or the service mediation layer translate the input into a generic URI for calling the RESTful implementation layer which in turn does the mapping internally to the required operational version of the URI. This improves the durability of the service interface layer or the service mediation layer. In addition, it provides a mechanism for doing internal operation overloading and polymorphism.

Regardless of which option is chosen, I would think that embedding RESTful services in the implementation layer would be a good option while not exposing painfully abstract behavior to the business user who is used to seeing more of English like business syntax.

Here is a link to another blog of mine on the topic in the context of Service Orientation!

Even though I agree that there is a place for REST as it may simplify (by providing a finite set of operations) there are infrastructural capabilities that do not exist with the REST and without these the business agility piece is going to be hard to deliver. Here are some additional thoughts on the topic that an enterprise architect may have to resolve prior to embarking on a RESTful implementation.

1. Identify which layer deals with the interpretation of the get(Resource) call? Is it the service provider (SP) interface or is it the service implmentation layer?
2. Would the SP be dealing with versioning of the Resource thereby creating a grand facade that does internal resource mapping via a URL redirect style substitution? This would be one way of protecting the service consumer (SC) and to accomplish backward compatibility.
3. Identify how the SC gets the handle or URL to the "right" resource? How does it "know" what to expect from the Resource and how does it "know" to express its' semantic expectations for the SP Resource that is being called upon? Does it do a "Get(SearchByCriteriaResource)" call with semantics embedded in the Request Resource?
4. Is the get(RightResource) another call and somehow the metadata held in the response Resource is expected to have the "right metadata" for each of the provider Resources (that meet the SC search criteria) in order to help the SC choose the right SP? If this is the case, then following this invocation would be a secondary SC to SP interaction to fetch the chosen SP Resource.
5. Identify if there would be a standardized URI with search criteria specification. This might be good to define up-front to enable the support of JINI-like or SaaS like interactions?
6. Identify and define how one would deal with asynchronous interactions? Would it be via the use of " ack/ no-ack post notifications" and "post response Resource" calls? How would this be to handled? The SC would have to provide a pre-set "call back" Resource identifier to which the SP would have to reply back to?
7. Would existing messaging infrastructures still work as most of them have proprietary protocols or else they support JMS/ RMI?
8. Identify a URI for brokering of the two-part synchronous post HTTP call to. This would be for enabling the pseudo-asynchronous interaction semantics. Is the centralized broker URI all that is needed to broker the two part synch call for faking the Asynch interaction semantics especially if the architecture does not have to deal with message delivery guarantee? I assume "post" might have void return type communication model to enable releasing of calling address space resources?
9. Would one have to leverage any "Registry" or ESB like middleware or "service grid" like infrastructures? If so, how does would resource proxying work? Would the proxy Resources’ "forward" their calls to the right Resource?
10. Would this mean that the middleware and mediation layer products need to know how to deal with this type of an interaction semantic where the "Registries", ESB, "service grids/ service marketplaces" are all metadata-driven Resource proxies with built-in support for REST - as opposed to service proxies as would be the case in SOA style architectural models.

Thanks in advance for your feedback.

surekha -

Key Best Practices - What is Service Orientation?

What is Service Orientation?

1. It is the ability to use of simple business syntax for defining and discovering business services whose interfaces encode business behavior using the business language
2. It is the ability to define a business service policy that governs the avaialbilty and reliability aspects of the business service

Service Orientation most of all is about the use of simple business terms defining business offerings that execute business capabilities. There is no place for the use of architecture terms, technological jargon or even as much as a mention of the web services or the WS-* stack in this definition!! The need for web service standards, SOA infrastructures technical stacks including HTTP, URIs or SOAP/ HTTP and WSDL enter the picture only when talking about the how to achieve Service Orientation, not the what is Service Orientation!

The technology is undoubtedly important provided an enterprise has the discipline of Business Architecture in place to define its' business service portfolio. In addition to this, the business service portfolio represents an enterprise' unique interpretation of Service Orientation and this enables the enterprise to deliver the strategic capabilities that are sought by the business. These strategic business capabilities are delivered as business service offerings that lead to market differentiation opportunities.

Without undertaking an effort to leverage in the Business Architecture discipline an enterprise has IT investments in technology of ESBs, the Service Registry and Service Repository etc. but the Web Services that are published look more like API calls. These API like web service interfaces will befuddle the business and will not help the enterprise realize the full potential of "Service Orientation". As would be noticed Service Orientation is built right into the concept of Service Oriented Architecture (SOA).

WOA and SOA are both architecture models that speak to the implementation models for achieving "Service Orientation".

WOA is a way of taking the REST-like service interface concepts that involve the invocation of standard operations that act upon simple payload identifiers and are transmitted over standardized internet protocols. The WOA paradigm embeds complexity in the "message format" or the payload. In other words, WOA prescribes a clean architecture model that enables an enterprise to quickly expose services with information constructs represented as URIs (instead of complex XML Schemas). Furthermore, the interfaces are defined as simple HTTP commands of "GET, POST, PUT and DELETE". It must be reiterated that the key point of the WOA model is to abstracted away complexity behind the simple interfaces and URIs. This aspect of simplifying the interface is very much in keeping with one of the principles of Service Orientation. However, this model may not really meet the other tenet of Service Orientation i.e. interfaces are represented using business vocabulary and are codified using business syntax.

SOA on the other hand is an architecture paradigm that leverages a more complex message format (that is defined as XML Schemas) with the interface definition that is could be deemed more business-like. Here the complexity of the message construct is codified using XML Schemas. These XML Schemas encode the business-document structural rules but do not expose the complexity of the business rules. So to some extent SOA meets the other Service Orientation tenant in keeping service interface definitions more business-like long as the interface definitions are driven by business architecture process.

WOA or SOA do not by themselves address the governance and monitoring of the business service policy. SOA infrastructure is needed to address service policy monitoring, service clustering and scalability concerns. In addition, the use of service infrastructure components such as ESB service-mediation flows enables IT to achieve location transparancy of business services that indirectly influence business service availability. It must be remembered that SOA infrastructure products are important to the business only in that they help meet the business’ service availability needs.

Just because the business does not need to be exposed to the implementation details does not mean that IT has the luxury of glossing over how the enterprise deals with transaction management rules and the incorporation of the ever changing regulatory business rules. It is just that these complexitites are left up to the technical and operations experts in the IT area.

An important aspect in talking about the maturity of SOA based technologies is that most commercial SOA infrastructure component vendors are striving to comply with the WS-* standards. This level of standardization promotes interoperability of the SOA based infrastructure components even when purchased from multiple vendors. The effect of this interoperability is that it finally affords IT a chance to now purchase packages to implement these complex implementation behaviors (such as transaction managment, service monitoring, service management and deployment, regulatory and security business rule interception etc.) as opposed to having to implement these complex implementation behaviors from ground-up. Now IT has a way of aligning its' business serivce delivery model to achieve the business need at the speed of the business instead of having to deal with infrastructure and integration issues.

Whether a WOA model or a SOA model is used to define and deliver the business capability to the business does not impact the concept of Service Orientation. The use of the WOA/ SOA constructs do not add a competitive business edge. They are just best practice architecture models that add to the reliablility of the service delivery aspect.

Your feedback is invaluable.
Thanks.
surekha -

Semantics in the realm of SOA and Services

The following whitepaper of mine delves into the topic of semantics in the realm of SOA and services.


http://www.soablueprint.com/yahoo_site_admin/assets/docs/SUITPOpinion_Roles_of_Semantics_in_Services_and_SOA.15781752.pdf

An area that I have not explored in the whitepaper is the usage of semantics to “discover” service offerings that are needed to deal with the "out of ordinary" alternate execution paths. It is true that today most of our processes are deterministic and follow a known or predetermined execution path. Any deviation from this deterministic execution path is typically handled by raising an exception or by routing the deviant information to a work flow component for human intervention. In future however, “runtime dynamic discovery” of services could be based on semantics that embed service ontology and process based state machine metadata - all of which work together in a business area to enable finding dynamic alternate execution paths that could be invoked to resolve “out of the ordinary” and exception situations.

As always your feedback on these thoughts and on the whitepaper content are invaluable.

Surekha -

Enterprise Service Bus vs. Service Component Architecture

My attempt is to present a simplistic view of my take on the purpose of an ESB (enterprise service bus) and SCA (service component architecture). This entry is in response to my esteemed colleague's commentary on "A decision makers concern about ESB " where in the two concepts of ESB and SCA were being compared. I must add that I am in complete agreement with Yogish on the fact that ESB vendors are adding to the confusion of what is and what is not part of the "core feature set" of an ESB.

My own opinion (misplaced it might be!!) is that ESB vendors tend to bundle a lot of "service creation"utilities and "adaptors" into the base ESB stack, in an attempt to achieve "product differentiation". This very act causes ESBs to become both ridden with "service behavior" which may also lead to "slow performance" of the ESB based mediation. The reason being that it becomes very difficult to performance tune an architecture component that is doing multiple things - it breaks the rule of "high cohesion" in doing "service mediation" and "service behavior execution"!!! The former is a high through-put pass through activity while the later could have fairly intense computation business logic or data processing logic.

Now back to the ESB vs. SCA topic.

ESB - promotes the abstraction of the service provider from the service consumer. Here it is assumed that the goal of exposing the service provider interface is to enable the reuse of the service provider business behavior. ESB' core theme being service mediation.

SCA - promotes the abstraction of wiring in of the external resource dependancies and external transport protocol bindings from that of the core and reusable business behavior of the service. Here the assumption is that the component parts of service provider can be re-assembled in various combinations to assemble services that are invoked from multiple consumer contexts. SCA' core theme being service assembly.

ESB enables service interaction/ mediation from multiple service consumer contexts while SCA enables service provider component parts to be assemble for "servicing" multiple service consumer contexts.

In both cases, the attempt is to decouple service usage from reusable service behavior.

Your comments on this topic are invaluable.
surekha -

Why you need a stated "service versioning policy"?

I felt I needed to expound on the topic of service versioning after I heard concerns from a few CIO level executives about needing to "support" multiple service versions. The reason stated was that maintaining multiple service versions was expensive from both a hardware and on-going support perspective.

I completely support this opinion in that if left "unchecked" - read without the right level of governance - this could turn into a maintenence nightmare (not to mention a regulatory hazard especially if the regulatory policies are not implemented uniformly across all versions). But then this is a case for having governance and not an argument against supporting service versioning policy.


I attempt to outline some reasons for defining and publishing a "service versioning policy" to support both the business need and to create an efficient resource-utilization operating model. A carefully thought out strategy of service versioning that is rigousously "governed" allows us to do some of the following:

1. support multiple versions where in business functionality may be altered based on the operating business context or business area or industry vertical
2. support multiple versions based on the user community needs (for example, invoking a service that incorporates work-flow rules vs. needing a service that is invoked as part of an automated process level interaction)
3. support multiple versions that allow an enterprise to reduce the risk to the business when introducing new business behavior or new business process optimization changes
4. support multiple versions of a service that allow the enterprise to incorporate and interject additional authorization, entitlement and audit related rules that change based on the business context and/or business user role
5. support multiple versions that allow the system administrators to provision server resources appropriately based on the QoS/SLA needs of the service consumer and the service contract definitions.
6. support multiple versions of a service to isolate more expensive business behavior calls to specific versions so as to reduce impact to all of the other consumers (For example, a version with externalized business rules while the business is "testing" a new rule would be less performant than a version where the business rule is ratified and encoded into the business logic layer. In this example the business behavior is externalized making the service version more flexible but this makes the service less performant)

Finally, I think that service versioning can be used as a strategy to both support the business need and to optimize resource utilization that in turn make consumer interactions more efficient. Thus, the "appropriate service version" is invoked based on the "right" business context.

Your comments on this topic are invaluable.

Thanks.

surekha -

Key Learnings : The End of the Application

This is a blog post in response to Todd Biske's posting on "The End of the Application" which can be found at the following URL.
http://www.biske.com/blog/?p=347

******************************************************************************
I am in complete agreement with you Todd, about needing a retirement plan for the term "application"!!! I share your sentiment about how this ubiquitous term reflects not only the siloed solution that gets built but also in how this term influences the way in which the business need is translated into requirements. Case in point is the way in which I have heard business analysts phrase their questions to the end-user. For example, "How do you expect to act on the information the application provides?" as opposed to asking "How do you or your user community make decisions based on this information?" OR "What is the business context (business rules, policies,regulations etc.) that influence the information?". The business analysts sould divorce themselves from thinking about the application as the "sole repository" of the information and it's current limitations. Analyzing the business user needs and the decision making process might help them discover that these business rules may have already been implemented elsewhere and that the "application" is but one component needed in delivering the targeted business capability. Sadly,this is not the case.


Part of the reason for gravitating toward point solutions and monolithic deployments is the sense of control the solution owner has on the entire solution stack and the confidence factor of knowing that a customized deployment has a better chance of meeting the response time SLAs. It is my hope that the availability of robust standards based deployment platforms may help resolve some of these latent fears. The key however is to recognize that the usage of the term application does influence the final implementation of the solution. Given this, there is ample justification for doing away with the term "application" as it may be too limiting in the realm of today's highly interconnected enterprise.

Doing this may lead to the required behavior change that is needed to move away from the creation of monolithic applications. As suggested by you Todd we may need to be more conscious of the term we use and this could be a simple matter of using a prefix to the term application or it may be that we have to throw the term away altogether. I propose using a prefix of "composite" to qualify the term application to get this desired change in behavior!!


The term application should be semantically equivalent to "applying" a
unique business context to the usage of a common asset such as a business service, legacy system or a business process. This "application" (note,the verb) of value-added business context is focused on delivering a crucial business behavior that offers competitive advantage to the enterprise. With that definition in mind, the term "application" may get a lease for life when used within the purview of the concept of "composite application" that suggests "distributed solution assembly" instead of "monolithic solution creation". Composite applications are mechanisms that help deliver novel business capabilities that are typically translated to"novel" assembly of reusable business services where by the composite application layer performs business context value-addition.


This assembly of business services within a composite application (as opposed to the creation of a monolithic application) enables the enterprise to realize "speed to market" gains from SOA. It also enables an enterprise to move to a model where the enterprise obtains base business services (off the shelf or SaaS style) and uses the composite application layer to apply the enterprise specific business context with the idea of offering distinct flavors of service offerings; thus moving away from the "creation" of solutions to "provisioning" of solutions. (It is worth noting that composite applications are not just deployed in the business layer but also
in the presentation layer in the form of Web 2.0 style mashups.) Finally, I feel that the only way we can move away from the mode of "monolithic point solutioning" to a mode of "assembling solutions" is by making the move to the concept of "composite application" as it breaks the existings emantics associated with the term "application". Following this mindset switch we may be able to completely replace this term as knowledge workers develop a comfort level in both the efficacy of "assembling solutions" at the same time that the vendors build up deployment platforms to help IT meet the stringent SLA needs that are demanded by the business.
***********************************************************************************

Key Learnings: Drawing parallals between Design Patterns and the priniciples of SOA

In looking through some of the Object Oriented Design Patterns, I found a curious parallal between the goals of some of the "Structural Patterns" and "Creational Patterns" and those of SOA.

The following are the two key principles that are most striking:
Principle 1: provide a clean interface or an entry point and hide all of the details from the caller
Principle 2: provide a layer of indirection between the interface and that of the implementation

It is interesting to see how these "design level" goals re-appear in the realm of architecture with changes only to the scale of applicability.

Principle 1 is supported by design patterns such as Facade, Abstract Factory, Builder and the Factory. Facade hides the behavioral details of the implementation while the others hide details of the details of the creation of a resource or assembly of a resource as in the case of "Builder".

Priniciple 2 is supported by the Bridge pattern that allows the plugging and replacing of any implementation long as the interface is supported.

The idea is not to arrive at a comprehensive mapping of the principles and design patterns but just to do enough to lend support for the core theme being presented here. This leads to only one conclusion, SOA does not preclude the use of good OOAD techniques. SOA just takes these concepts further into the realm of business architecture and business analysis to insure reusability and enterprise wide applicability of the IT asset, in this case a "service". SOA also has been lent support by the standardization of platform agnostic protocols and message model (such as SOAP/ HTTP) and the use of canonical business models (such as XML Schema backed XML).

Your comments are invaluable.
surekha -

Extending the use of Service Enabled Interfaces for dealing with Business Events

Typically Business Services are considered to be invoked in a synchronous request-response mode. Business Applications are supposedly called into action by human actors who leverage user interfaces. On the other hand, Business Processes are typically assumed to be triggered by Business Events.

However, it is possible to extend the use of Business Services and/or Business Applications to deal with Business Events. This enables the reuse of existing business logic and business rules in order to deal with new buisness interactions which are introduced into the ecosystem without needing to undertake a large service implementation effort.

For the purpose of this discussion Business Services, Business Processes and Business Applications will collectively be named as Event Response Interfaces. Business Events can be routed to the receipient Event Respsonse Interfaces by an ESB or a Message Hub. Business Events may also be sent to a Complex Event Processing Engine for aggregating, filtering or de-duping business events prior to routing them to the apporpriate Event Response Interfaces. Having said that, how does one extend existing Service Enabled Interfaces to trigger them asynchronously?

Architecturally, adding a Message Listener that in turn calls the Service Interface (such as seen in a Service Activator JEE pattern) or leveraging a Service Mediator (such as an ESB or a Message Hub) can allow Business Events to invoke Business Services or Business Applications outside of their regular request-response mode of interaction. The use of this simple architectural strategy allows Business Events to be processed by existing well tested Business Services or Business Applications.

One additional point to keep in mind in extending the interaction context from synchronous mode to that of an asynchronous mode is with regards to dealing with business exceptions, faults and business responses. In a synchrounous communication mode both of these are returned to the invoker of the business service or the business application. In an asynchrounous mode there is a disjointed execution context where in the invoker is not blocking on the call and waiting for execution control to be returned. The execution path may or may not be re-established.

To deal with this asynchrounous communication model the service response and faults may either be logged in a common repository or may be returned back to the invoker by the service mediator. The service mediator may re-establish the execution path with the invoker. The service mediator such as the ESB may refer to the Registry to find the originator or may leverage the correlation identifier and originator address in the payload (a provision made by the WS-Addressing spec. via the use of wsa:From metadata element).

Regardless of which architecture component is used i.e. an ESB, a message hub or a message listerner that intercepts the Event Response Interface the result is the reuse of the service behavior. In addition to reusing the business service applying this simple service extension technique also enables leveraging existing investments in infrastructure components such as the ESB or message hub thus helping achieve business agility.

Please provide me with feedback as to whether or not you have leveraged existing IT service assets to introduce new communications pathways.

surekha -

Key Learnings: Canonical Models vs. Domain Models

In this blog my attempt is to provide some definitions and in turn to get feedback on the differentiation between the parameters traded between a service consumer and a service provider (Canonical Models) vs. the parameters traded between the various internal architectural layers of an application (i.e. the Domain Models).


Canonical Models
Example of canonical models - Customer Preferences, Supplier Profile , Vendor Service Offerings and Accepted Service Bid

1) Shared as parameters for trading between service consumers and service providers
2) In SOA context these are typically transformed into syntax agnostic and platform neutral XML
3)Business Service usage context is associated (for example, information about a quote vs. a final offer that has legal implications is embedded in the canonical model)
4) No real business logic is embedded giving service provider the freedom to alter the behavior without breaking the service contract
5) Cross-business domain relationships formulated by representing multiple business concepts are included so that the consumer can avoid making multiple remote calls

Domain Models
Example of domain entity - Purchase Order and Customer

1) Shared as parameters between persistence layer and business logic layer
2) May or may not be expressed as XML especially if traded within the business domain applications/components
3) Domain context and domain internal relationships are captured
4) Domain specific business logic may be encapsulated as the model is traded within the business application or within the business domain. Also, the business logic in the domain entity has relevance internally to the business domain alone.
5) Single business context and only relevant amount of information in terms of attributes and relationships are represented so as to avoid hydrating a large volume of information into memory.


Thanks in advance for your feedback.
surekha -

Business Process vs. Business Service

The purpose of this blog is to get some validation for how I look at Business Processes vs. Business Services. In simple terms, I differentiate between these concepts in the following manner:


Business Process
1) Triggered by business events
2) Issues a series of business actions that could involve the invocation of one or more granular business services
3) Interactions involve execution of a business activity that is representative of a value chain or is a core part of a value chain in the enterprise

Business Service
1) Typically triggered by a request from an online consumer or a single business transaction
2) Issues a single business action for the most part
3) Interactions involve the execution of a single business function or an inquiry against a single core business entity

Your feedback is invaluable.
surekha -

SOA:Many Things!

SOA: Many things

SOA is not one but many things! Recall the story of blind men and the elephant; each one comes up with a different interpretation. Our individual perceptions and role in an organization is not an adequate basis to describe the entire SOA beast. Agendas of individuals or organizations defining SOA make it even harder to fathom through the myriad. For instance, for a vendor with a product line of applications, SOA is just web services. For a technologist, it is an IT issue and all about SOAP. SOA is about legal contracts for the general counsel, or delivering a business capability for a CIO.

In short, SOA means many things to many people. Yet we are not blind and should not be constrained by a narrow view. Thus the overall definition of SOA must arise from collective nature of the enterprise, with input from its constituents.

Let’s consider a situation involving the return of unused goods. In this typical reverse logistics scenario, understanding of a service varies from technologists to non-technologists, each with different priorities. For a technologist, a service is a network-enabled software component, which provides a specific capability. For a non-technologist, a service is some form of offering supplied by an organization that receives merchandise return requests, sends directions for shipment, updates inventory and applies credit to the customer. A customer service representative will make sure that customer requests are understood and s/he gets the information for the return. Shipment and receiving will be your focus if your role is operations. Accounting will be more interested in applying appropriate credit to the customer. While each participant has a defined focus, overall goal is to establish a means of delivering value to the customer.

This complex situation involves Customer Relation Management, Supply-Chain Management, and Enterprise Resource Planning. SOA solves this complexity by

• Decomposition
• Aggregation
• Orchestration

SOA decomposes this business process into well defined discrete services: some are automatic, some are manual, some short-lived and near real-time, some are long-running requiring human-intervention. Services may be aggregated and then orchestrated to create additional services depending on the granularity level of decomposition.

After all, a service is a unit of work done by one entity in order to meet the need of another entity. The primary entities in SOA are the service provider and the service consumer. At any point, the roles may be reversed, consumers becoming providers and vice versa. The consumer enjoys the value created by service without the ownership of the specific costs and benefits.

Service interactions, however, must be defined unambiguously for both entities to successfully communicate. Each interaction between the participants must be self-contained in a service contract, focusing on a specific capability. A greater synergy is achieved by the orchestration of these services in a coherent manner.

SOA embraces the fact that each entity may have a drastically different environment. Distinct systems or applications deployed at various agencies can interoperate because SOA makes as its pinnacle the use of ubiquitous XML which poses no limitation by platform or language. Choosing JAVA, .NET, or a mixture is not a headache anymore! By placing emphasis on business activity rather than implementation details, SOA can make that crucial inventory drop arrive at a warehouse manager’s Blackberry™ to shore up just-in-time measures. By adhering to the service contract, an organization can easily swap out its implementations or move to other locations. A service for sending out information to customers, for instance, may be reused by multiple departments.

SOA is more than a buzzword. It is the way for organizations to rapidly develop capabilities, to collaborate and meet the expectations of their stakeholders. SOA’s agility makes it an enterprise business strategy, however.

SOA may also be DOA, dead on arrival, if we narrowly assume that it is just a bunch of web services, a product or a panacea for integration nightmares.

Some organizations get their feet wet with a few services by creating prototypes and exploratory projects. This is a good point of entry for the technologists but the business executives need to start thinking big at the onset. For businesses, a concerted effort needs to be made to launch a SOA lifecycle at the enterprise or the segment level. A typical SOA lifecycle includes three stages:

• Initiation
• Roadmap development
• Plan execution

For continued success, the lessons learned in execution should create a feedback loop to the roadmap.

SOA initiation needs both business and IT to cooperatively decide on the business process that will be built or replaced. Creating the team and setting a project plan is necessary. Roadmap development establishes the SOA principles and reference architecture for business and technology (application, data, and infrastructure). Governance will guide with policies, measure effectiveness, control for compliance, communicate to inform all parties, and above all, empower for better decision making. Program management assures that the project is on time, on budget and on scope.

The SOA lifecycle is about implementing SOA enterprise-wide. It gauges smaller pilot projects to be on par to address the need of the organization and paves the way for properly scaling up. Once business and IT decide on the services, avoiding DOA requires establishing a Service Lifecycle to manage individual services.

Service lifecycle spans the activities from inception to retirement or decommissioning of a service. There are three stages:

• Requirements and Analysis
• Design and Development
• IT Operations

Following a service lifecycle assures accurate capture of business requirements, development of IT solutions that match business requirements and deployment/maintenance of services that provide business capabilities encapsulated in the requirements.

Starting a SOA lifecycle, or for that matter, choosing the candidate business processes along with governance, is not a small feat. A misfiring engine can cause headaches in the long drive. Choosing a vendor in this endeavor may be the biggest investment an organization may have to make in climbing the mount SOA.

Note that SOA is neither a product nor a single solution, but rather a comprehensive business enablement with IT elements. Make sure that your vendor is capable of bringing together business and IT stakeholders in a face-to-face collaborative where you, as an organization, not the vendor, make the decisions. After all, who knows your organization better than you? How rapidly will your vendor comprehend and assess your business processes to enable you to sail into service decomposition? Think big and ask about the end game. Establish governance and a program management. Spot the need to change early, especially organizational changes. Start small with services representative of your environment then you are ready to scale fast!

Burc Oral

http://www.linkedin.com/in/burcoral
http://www.soa-consortium.org/

Best Practices - Is Canonical XML dying? - Reply to IC Blog

This posting is a reply to the blog posting on Canonical Model http://www.icmembers.org/blog/zaki/2007/10/5/is-canonical-xml-dying#comments



In general, I agree with the posting on Canonical XML discussion. I wanted to make a couple of additional comments on industry-vertical specific Canonical models and the benefits they offer to those enterprises whose interactions leverage standards based information exchange.



a) implementations leveraging package products that exchange canonical models expressed in XML allow ease of upgrading and/or switching between package products without paying a huge upgrade penalty. This is so in the case of functionality related upgrades that do not alter the service definition and/or the information exchanged. Also, this makes decommisioning of redundant packages easy when this becomes necessary during acquisitions long as the package implementation are all standard canonical XML based.



b) partner collaborations also become easy, reliable and robust when collboration is based on the use of common industry vertical XML representations. Adding more trading partners becomes easy when using standards based information representations and this in turn leads to predicatable interaction patterns and information exchange.



c) documentation that refers to common service interactions that form the basis of that industry vertical when captured in training artifacts enable standardized interpretation of the material. In addition, given that this type of training is transferable between enterprises belonging to that industry vertical sharing personnel among trading partners is also made simpler.

Your comments on the canonical model usage is appreciated.
surekha -

Key Lessons – The new and improved Enterprise Architecture Group

When invited to start the Enterprise Architecture Group a couple years ago my only thoughts were how to technologically implement SOA and EDA as the core architectural principles. However, soon enough came the realization that without having access to those who had sound business knowledge and those that knew the core business domains, value streams and the strategic business capabilities it would not be possible to effectively deliver SOA or EDA.


Service orientation of business behavior encapsulated in business applications and the definition of business events that initiate key business processes both require enterprise-wide understanding of business behaviors and business activities. Furthermore, the ability for identifying business services and/or business events comes from being able to abstract business needs that cross business domain boundaries. These skills can be found in a group of individuals being called business architects – a unique group of individuals who can translate the strategic business needs into an action plans that show how to enhance current business processes that support core value streams.


This led me to find resources that could analyze business processes (a.k.a. Process Architecture discipline) and resources that could analyze business information-flow, the value-addition pathways (where by business information is enhanced as it is exchanged across multiple business domains) (a.k.a. Information Architecture discipline). Both sets of these resources have allowed me to embark on true SOA style service definition. So the “new and improved” Enterprise Architecture Group has Process Architecture, Information Architecture (or what the industry calls Business Architecture) and Technical Architecture representatives.


Finally a note about the Technical Architects; my opinion is that, standards are vital but then if the Enterprise Architecture Group is unable to demonstrate the right use of standards in a true production environment this group will be left with very little credibility or be deemed the “ivory tower” group.


Having a cross-functional group of resources has made the Enterprise Architecture Group to be a well rounded group. The Business Architects of the group are in dialogue with the business as they can speak to the business in its lingo while the Technical Architects of the group are able to work with the developers as they are able to demonstrate the use of technology for building and delivering stable, performant and reusable services. The end result is that the individuals of Enterprise Architecture Group are openly invited to the table for consultations and recommendations.


I would love to hear of other organizational patterns/ formats that have enabled implementation of SOA and EDA.

Thanks.
surekha -

Best Practices - Mashups and the importance of Foundation Business Services

It seems like interactive web based social computing experience and SOA style services are not really thought of as related. However, when mashups are incorporated into the realm of the enterprise the need for ease of collaboration as well as accuracy tend to matter and this is where enterprise worthy services enter the picture.

In addition, having access to a registry is key. This SOA based infrastructure component becomes the central point for discovering loosely-coupled services that encapsulate core business functions and business semantics. To fully enable the knowledge worker in the selection of services for mashups, the business user community not only has to be included in the definition of the service but it has to be involved with the process of publishing these services to the registry as well. The reason being that business keywords that are used in describing the service are identified and entered when publishing the service to the registry. This level of cooperation and collaboration with the business user insures that the services are described using business language and grammer.

Finally, cross-business service payload validation behavior has to be put in place to prevent users from creating mashups that combine incompatible services. For instance, only services sharing related business contexts or those that return information at the same level of granularity should be allowed to be pulled together. Tools/ IDEs of the future that aid in the creation of mashups may be able to interrogate the categorization of business oriented services and informational metadata descriptors published into a registry or a repository to make a determination of inter-service compatibility. In the near term however, an enterprise will have to resort to governanace and testing policies to allow compatible types of services to be mashed up.

As can be seen in an enterprise context users can gain the capability to perform effective mashups when they can leverage reliable foundation services to access enterprise information systems. This way IT can enable user self-service by just making core services available to the knowledge worker and enhance business agility.

Your feedback is invaluable.
surekha -

Key Observations - Service Reusability - Does it really work?

From my experience, on SOA projects reuse was mostly accomplished with technical services/ infrastructure services. However, it is the business service reuse that we are struggling with as the Lines of Business (LoB)/ business units feel that their sub-set of the business logic is very unique. The problem is that most of these business services are rendered non-reusable due to the fact that they include both business logic and process sequencing logic into the same code base and it is the later that is unique to the LoB and not the business logic. Also, we have achieved greater degree of reusablitiy in terms of data access and informational services and not so much business function services that are a higher level class of services.

I would like to hear about what your experience has been with regards to service reusablity. Have you been able accomplish this.

Thanks,
surekha -

Key Learnings - Profile of a Business Service

My experience has led to the creation of the following service based postulates. These were applied by me on major retail SOA projects.

You can find these and other laws posted on the SOA Blueprint site. http://www.soablueprint.com/soa_laws

1. A service should expose a single business function or business process in it’s' entirety
2. A service should not make assumptions of the validity of the request and should validate the request for accuracy including checking that the requestor is authorized properly prior to honoring the call
3. A service should delegate to the right enterprise resources to completethe business function
4. A service should be responsible for notifying and dispatching the right requests/ information to all related services and/or applications on behalf of the requestor
5. A service should be responsible for any call chaining, sequencing and aggregating of the requests and the intermediate responses in order to completely process the consumer’s original call
6. A service should provide business information as a canonical documentthat is consumable by both human actors and system actors
7. A service should return a response that has the business context built in along with the response to the question asked by the requestor
8. A service should rely on enterprise worthy metadata information including the application of any algorithms, enterprise-wide business rules and strategies in order to process the requestors' call

Your feedback is appreciated.

- surekha -