SOA Practitioners’ Guide
Why Services-Oriented Architecture?
Painting by: Surekha Durvasula
Contributing SOA Practitioners
Martin Guttmann, Principal Architect, Customer Solutions Group, Intel Corp
Ashok Kumar, Manager, SOA Architecture, Avis/Budget
Tom Mitchell, Lead Technical
Burc Oral, Individual Contributor
Tom Sedlack, Enterprise Architecture & Engineering, SunTrust Banks, Inc.
Dr Harsh Sharma, Senior Information Architect, MetLife
Sankar Ram Sundaresan, Chief Architect e-Business, HP-IT
Prasanna Deshmukh, Director of Architecture, WebEx Communications
Noam Fraenkel, CTO IT, Mercury Interactive
Steve Jones, CTO Application Development Transformation, Capgemini Group
Brenda Michelson, Principal Consultant and Analyst, Elemental Links, Inc.
Ashok Nair, Management Systems Analyst, EAI, Information Technology Services,
George Paolini, Consultant, Georgepaolini.com
Pendelton, Executive Director, SOA
Annie Shum, VP SOA Strategy, BEA
The authors would like to acknowledge the many organizations and individuals that contributed portions of this document, performed substantial editing, or who provided reviews and feedback. In addition, the authors would also like to thank BEA Systems, Inc. for providing the infrastructure and platform for developing and presenting this guide.
SOA is relatively new, so companies seeking to implement it cannot tap into a wealth of practical expertise. Without a common language and industry vocabulary based on shared experience, SOA may end up adding more custom logic and increased complexity to IT infrastructure, instead of delivering on its promise of intra and inter-enterprise services reuse and process interoperability. To help develop a shared language and collective body of knowledge about SOA, a group of SOA practitioners created this SOA Practitioners’ Guide series of documents. In it, these SOA experts describe and document best practices and key learnings relating to SOA, to help other companies address the challenges of SOA. The SOA Practitioners’ Guide is envisioned as a multi-part collection of publications that can act as a standard reference encyclopedia for all SOA stakeholders. .
This document is intended for the following audience:
· Business and IT leaders, who need to start and manage an SOA strategy across the enterprise/LOB
· Enterprise Architects who need to drive the vision and roadmap of the SOA program and the architecture of each implementation that falls under it
· Program Managers who need to manage a portfolio of sub-projects within an overall SOA business strategy
· Project Team Members, who need to map dependencies and develop a timeline that meets the business expectations
· Vendors who provide solutions and tools for new business capabilities to the business and IT
· Standards bodies which need a better understanding of use cases of how business and IT plan to leverage technology to meet their objectives.
This document helps readers to:
There are three separate parts that make up the SOA Practitioners’ Guide.
This document is Part 1, Why Services-Oriented Architecture? It provides a high-level summary of SOA.
Part 2: SOA Reference Architecture provides a worked design of an enterprise-wide SOA implementation, with detailed architecture diagrams, component descriptions, detailed requirements, design patterns, opinions about standards, patterns on regulation compliance, standards templates, and potential code assets from members.
Part 3: Introduction to Services Lifecycle provides a detailed process for services management though the service lifecycle, from inception through to retirement or repurposing of the services. It also contains an appendix that includes organization and governance best practices, templates, comments on key SOA standards, and recommended links for more information.
For several years now, SOA, or services-oriented architecture, has promised to deliver unprecedented flexibility and cost savings to IT, by defining a methodology for the use and re-use of software components and business processes. However, SOA is still new, and organizations are still in the process of learning how to implement it so that it fulfils its potential for intra and inter-enterprise services reuse and process interoperability. Meanwhile, would-be SOA practitioners encounter the challenges typical of a software methodology that is not yet supported by a fully mature industry. For example:
Traditionally, IT works with the business owners, who are influenced by application vendors. This results in IT strategies that are application or integration-focused. In addition, governance and funding models have pushed both business and IT stakeholders to do whatever it takes to meet a particular business unit or department need. This results in many “one-off” applications that may or may not be integrated into the existing architecture. Mergers and acquisitions introduce new software platforms and methodology to an already fragmented architecture, but IT rarely has sufficient resources to complete business systems integration. As a result, IT often ends up deploying multiple systems that perform the same tasks within an enterprise or business unit. Redundant infrastructure solutions for authentication, single sign-on, and data marts, as well as applications (packaged and custom), such as sales force automation (SFA), quoting, and order management compound the complexity and cost for IT. It becomes nearly impossible—and definitely impractical—to modify this portfolio to reflect a change in a business process or accommodate an acquisition.
In many organizations, IT groups integrated these siloed systems using a point-to-point or EAI approach that connected the application to both upstream and downstream systems. To track the transactions across the business process, IT propagated some key values across the applications—sometimes inconsistently—and created different operational data stores for each business unit to track key performance indicators. Then, to provide a seamless user experience, IT frequently built portal applications to connect to multiple backend applications, data marts, and master data.
Figure 1: Current Best Case
While effective from an architectural standpoint, such solutions are extremely complex and expensive to maintain or extend. These solutions are difficult to modify, so when the business asks for a change, IT is slow to respond. This creates a perception of conflict.
A business’s IT organization is critical to enabling the business to respond to challenges, such as competitive pricing, offshore suppliers, and other issues that arise in today’s rapidly changing global marketplace.
Business must be agile to survive globalization. When competitors can make the same product for less, a business needs to respond creatively—and quickly. IT is rarely given the opportunity to influence expectations and alternatives.
Even though corporations have record cash reserves, market growth remains anemic. Companies are trying to achieve growth through mergers and acquisitions. These put additional pressure on both business and IT organizations to consolidate and simplify disparate people, processes, and systems.
With growth slowing, businesses have concentrated on cost-cutting. IT has responded to budget cuts by outsourcing non-core functionality and adopting the offshore development model. While not ideal, this enables IT to free up some of its limited budget for development of new business capabilities.
This trend is
expected to grow exponentially over the next few years. Outsourcing enables
enterprises to focus on their core businesses and outsource non-differentiating
services such as Human Resources,
Enterprises have to comply with government regulations to stay in business. Some recent regulations, notably Sarbanes-Oxley, required IT organizations to review and in some cases retrofit all business systems. This diverted scarce IT resources and funding away from development of new business capabilities. Future regulatory changes could require further changes.
New technologies have the potential to create new business capabilities. But seizing new opportunities can also create new challenges, especially for IT, which usually has to figure out how a business will sell, supply, and bill for a new product or service. With resources scarce, IT must make tough choices between maintaining and upgrading existing IT systems and investing in new systems to create new business opportunity.
In most cases, LOB decision makers do not clearly understand the benefits of shared infrastructure and business logic. They focus on their immediate need, and IT responds, often developing multiple systems with almost identical functionality but conflicting algorithms that complicate any attempts at integration and consolidation. Many companies have multiple, incompatible solutions for authentication, single sign-on, and data marts, as well as multiple applications (packaged and custom) for sales force automation (SFA), quoting, and order management.
Corporate structures, too, often provide little incentive for sharing and cross team collaboration. This can lead to a project focus in IT, rather than a coherent business information strategy and business-based IT infrastructures.
At last count there were more than 56 standards bodies addressing various aspects of SOA. Their efforts are not coordinated, so the result is almost the same as having no standards at all. It’s up to individual IT groups to decide which “standards” to follow.
Business would like IT organizations to be more agile, but don’t want to pay more. IT organizations need resources in order to keep legacy applications running well, develop more agile and productive infrastructure, and add needed business capabilities. Traditional governance, organization, and project management models cannot resolve this conflict. The only way to meeting these ongoing business challenges is joint business and IT transformation.
Most business and IT executives agree on one fundamental business principle: their business processes differentiate them from their competition. For some, it may be the way they handle their supply chain; for others, it may be their ability to bring new, innovative products to market. Focusing on the business processes is important for enterprises to mature to a more flexible goal-oriented model. However, business and IT operations teams frequently differ in their approaches. For example, some business operations teams prefer to demonstrate “quick wins” to validate an approach, while IT operations prefer to build out the infrastructure. Fortunately, SOA offers both.
Figure 2: Business Value v/s Complexity
Figure 3: Conflicting Business & IT Priorities
Experienced SOA practitioners recommend the following approach for developing a roadmap to resolve the perceived conflict between the aims of the business and IT:
· Understand the business services: this is the critical first step towards successful adoption of SOA
· Develop an information strategy that identifies key performance metrics such as “reduce product defects by x percentage,” or “respond to all mortgage requests within 24 hours”
· Develop an SOA blueprint that articulates the business benefits and includes business principles, reference architecture, roadmap, and governance and organization guidelines
· Identify “quick wins” to demonstrate that, with SOA, IT can improve on its current average of 15-18 months to deliver new capabilities.
Once practitioners have taken these steps, IT organizations need to identify the infrastructure services required to deliver business solutions. Designing and building infrastructure services to support coarse-grained, loosely coupled, and standards-based services enables IT to be transform itself as the business requires. IT can then proactively or responsively deliver global solutions, with reduced application and infrastructure complexity, increased reuse of business services, and service orchestration capabilities.
Blue: Business Solutions Red: Services Infrastructure Grey: Business Processes
The above diagram shows both business solutions and the services infrastructure required to provide these solutions. SOA practitioners recommend developing the services infrastructure as required. IT should map the services infrastructure while developing the SOA roadmap, because this mapping enables IT to show the benefits of reuse, as well as demonstrate the agility for developing new or modifying existing business solutions. The following examples illustrate how a team might map business solutions to services infrastructure to solve today’s typical business challenges.
Provides a single portal for employees to perform all personal administrative tasks, such as address change, benefit enrollment, time and expense reporting, and employee on-boarding
Single view of the customer (SVC)
Provides an SVC across all business silos based on roles and information needs of the customer
Shared data services
Requires business process orchestrations
Business process management
The lifecycle of SOA is similar to that of an enterprise architecture, with an emphasis on business processes. Unlike an enterprise architecture approach, however, SOA requires both business and IT transformation, replacing traditional governance and organization models with a methodology that enables greater flexibility. Business and IT stakeholders need to look past their immediate areas of responsibility and partner to define the strategy and the roadmap to achieve their objectives.
SOA is a business operations strategy for leveraging information to meet the organization’s objectives, such as increasing overall revenue, boosting customer satisfaction, improving product quality, and enhancing operational agility.
In practice, SOA means different things to different people. To an IT Architect, it means the overall enterprise architecture definition and the process that enables IT to develop and deploy business capability rapidly. Architects seeking information usually go to vendor sites for product details, case studies, and best practices which they then compare to other publication and standards sites such as The Server Side, W3C, java.net, and OASIS.
To the LOB-IT liaisons it means the governance, organization, and process for project/program management–and the various business building blocks that could potentially be reused in order to reduce cost. The LOB-IT and CIOs typically attend CIO Forums or subscribe to journals that provide information from vendors or analysts.
For the legal team, SOA is a potential liability: they want to know whether the service is delivering information to partners, customers, and others outside the company firewall, and what regulatory issues and exposure might arise.
And finally, for the CIO, SOA is the
IT strategy for delivering business capability. What business functions will be
automated: which services, and what maturity, cost, and
The SOA lifecycle covers the stages of implementing SOA within the enterprise or LOB and is different from the service lifecycle, which is the process for managing individual services. The SOA lifecycle has three stages: initiate, develop roadmap, and execute plan.
Figure 4: SOA Lifecycle
First, IT and business must decide which business function and underlying processes SOA will enable, enhance, or even replace. This often takes the form of IT “pitching” a project to a business stakeholder, presenting the business and/or technical events that the proposed SOA-based strategy will address, and what benefits this new approach will deliver.
At this phase, the company establishes a project team, objectives, and timelines & deliverables. The objective is to create a roadmap that combines business and IT efforts, and that will be approved by an IT Board of Directors.
Following are some examples for this proposal:
Following is an example of a high-level project for initiating SOA. The proposed project is expected to last 11 weeks and each organization should modify the timeline according to their needs. The relative effort and the tasks would potentially be the same.
Figure 5: High-Level Project Plan to "Initiate SOA"
Best practices for initiating SOA include:
· Creating an actionable roadmap, one that both business and IT leadership are committed to
· Having the Enterprise Architecture team lead the effort
· Pulling together a project team including all key personnel from business, IT leadership, LOB-IT, operations, and partners such as systems integrators and ISVs
· Documenting clear business objectives, plans, and financial information
· Securing a balanced and unbiased opinion from a system integrator to identify the business benefits and develop the ROI
· Leveraging resources from ISVs, including product roadmaps and best practices
· Setting a reasonable timeframe for the project, typically between 8 and 24 weeks.
CIOs frequently sponsor SOA initiatives with an IT Board of Directors as the steering committee. The following business personnel should be included in the project team:
· Business Team Members
o VP, IT/Business Strategy
o VP Business Operations
o Representatives from each business operations team.
Their responsibilities include identifying, defining, and prioritizing business processes and benefits.
The following IT personnel should also be included in the project team:
· IT Team Members
o VP IT Application Development
o VP PMO
o VP Technology/Architect
o Divisional CIO/LOB-IT head.
Their responsibilities are to translate business strategy to IT strategy, develop an IT execution plan aligned to business priorities, develop reference architecture, identify architecture components, and estimate cost.
· Define how IT should align to the business
· Develop an execution plan for achieving the vision
· Create governance and organization model to execute the vision
· Estimate expected business benefits (ROI) for the investment
· Statement of SOA principles, as they relate to the business, application, technology, and data (for more information, visit http://www.opengroup.org/togaf/)
· Inventory of current situation, including business applications, owners within business and IT, functionality, support contacts, technology providers, data flows, and skills of current staff
· Recommended target state consisting of the reference architecture, gap analysis, roadmap, and technology required
· Execution plan consisting of application portfolio, implementation sequence based on business priorities, and mapping of infrastructure needs to provide business capability
The team needs to create a roadmap, which spells out the process for conducting an SOA assessment for the organization, developing the SOA principles, defining the reference architecture (future state), and making the transition from the current situation to the future state.
Following are the three key deliverables for this stage of SOA:
· Develop SOA principles: define the SOA principles in a clear and concise manner.
· SOA reference architecture: describe the “future state” for the IT organization. The SOA reference architecture maps out the technology “end-state” and as part of this stage, the team needs to define the business “end-state”. This will help develop the SOA roadmap for the enterprise/LOB.
· SOA roadmap: the roadmap should clearly define the phases for deploying business solutions and the infrastructure required to support them. It should also identify opportunities for quick wins to validate the benefits of SOA.
The execution plan describes how to execute towards the SOA roadmap. Execution has two tracks:
Teams need to review the SOA execution plan on a regular basis and update the roadmap whenever there is major change to the plan.
Some companies have run into trouble by rushing into SOA. Here are some of the most common—and easily avoided—SOA mistakes:
· Assuming that Web services is SOA. While Web services are an important foundation for SOA, SOA is an architectural approach, not a collection of granular services.
· Viewing SOA as the solution to all integration challenges. Companies must begin by determining which applications or integration efforts are well suited for SOA.
· Thinking that a shrink-wrapped “SOA solution” will jump-start the SOA initiative. There is no “out-of-the box” SOA solution. Companies must go through a careful process of selecting products that are interoperable and based on industry standards and architectural frameworks.
· Ignoring the needs of the business. If a team treats SOA as an IT project and creates a service that does not meet business expectations by delivering needed functionality with performance, reusability, and cost-effectiveness, the project will fail. SOA is not an IT project; it’s an enterprise-wide approach and requires enterprise-wide support.
· Committing to a platform. Adopting a platform-agnostic approach that is model-driven will deliver better ROI and flexibility in the long run.