Web 2.0 success stories driving WOA and informing SOA

http://blogs.zdnet.com/Hinchcliffe/?p=168

April 17th, 2008

The striking contrast between the stories that we’ve been hearing lately about the slow going of SOA initiatives in the enterprise and the vibrant and rapidly growing ecosystems similar to them on the consumer Web has been generating a lot of debate and discussion in the enterprise IT community recently. This discussion was brought into sharp relief when ZDNet colleague Joe McKendrick recently reported on Burton Group’s Anne Manes stating that it “has become clear to me that SOA is not working in most organizations“, based on a wide ranging study they performed.

It’s become clear that the SOA world will have to change some basic assumptions.This is just one data point of many recently showing the continued shortfalls we’ve experienced in trying to get our enterprise systems to work together in the ways that we would like. Organizations clearly want to leverage high levels of interoperability to seize new business opportunities, innovate on top of existing assets, and properly leverage the extensive landscape of software, data, and infrastructure that most organizations have accumulated in large quantities over the years. But we are still having a great deal of difficulty doing so and SOA investments are just not reaping the types of return on investments that most businesses would like to have.

Looking for answers on how to improve SOA

This has driven a search for new models since there’s little question that the core ideas behind SOA seem to be the right ones. Rather, it’s been how we’ve gone about designing and implementing SOAs that appears to be at the crux of the issue. As we look at the most successful examples of SOA actually working, we keep being drawn back to the Web itself, with companies such as Amazon and their highly successful Web Services Division (with hundreds of thousands of business consumers of their global SOA), Google and its numerous and varied open Web APIs from Google Maps to Google Data, eBay and billions of dollars in listings it generates through its public SOA, or the rise of applications like Twitter (which gets 10 times the use through its APIs than from its user inteface) and applications that are primarily used via their SOA presence. Then there is theincreasingly widespread adoption by millions of users of a sort of “visual SOA” with Web widgets and gadgets as well the rapidly growing story ofsoftware mashups, aka composite applications in the SOA world. There are many more SOA-ish success stories like this on the Web, but few in the enterprise.

John Musser’s ProgrammableWeb remains the best directory for finding all the APIs that Web companies have contributed to the Global SOA. Over 700 APIs are listed currently.

So if so-called Web 2.0 companies — which value participation almost above all else, both from consumers and organizations that want to integrate them into their offerings — are seeing highly desirable levels of adoption and significant ROI, how can this help understand how to improve our efforts in the enterprise? Most new Web 2.0 applications start out life with an API since getting connected to partners that will help you grow and innovate is a well-known essential for success online today. Despite years of SOA, we still don’t focus on consumption and openness as fundamentally essential characteristics to building an internal partner ecosystem that have beat a path to your door to use the services you are offering to them to build upon.

One big issue, as I’ve written about in the past, is that enterprises are often very much unlike the Web. Many of the aspects that make the Web successful (simple flat link structure, billions of deep interconnections, a handful of simple and easy to process data formats, etc) just don’t exist in the enterprise with it’s wilderness of relational databases, proprietary applications, and silos of every description, despite some success in adding a traditional SOA layer on them.

Turning to WOA as an exemplar of good SOA

Dana Gardner recently wrote about here on ZDNet and respected SOA expert David Linthicum covered on Infoworld, the integration and ecosystem models that are working on the Web are increasingly being discussed in the SOA community as potent examples of how to implement SOA with better results. As WOA examples get better and better, it’s emerging solutions to many of the issues we have in getting robust SOAs built with high levels of adoption and measurably better business outcomes.

As we extract the positive outcomes of thousands of startups on the Web that have tried virtually every integration and composition scenario imaginable, we’ve begin to see the broad outlines of what works. And these new models intrinsically take advantage of the important properties of the Web that have made it the most successful network in history. This extraction of what works best has begun to be called Web-Oriented Architecture (WOA), which I’ve explored here in the past several times and was coined a few years ago by Nick Gall. The concept of WOA has evolved as we’ve learned more lessons about the new approaches working on the Web.

Read a more complete and technical description of Web-Oriented Architecture (WOA).

One of the challenges of WOA is that it’s an emergent set of best practices that we can independently observe and verify on the open Web. It’s not a formal set of standards but an set of approaches and delivery techniques that seem to be work extremely well and achieving most of the major goals that we originally set out for ourselves in the SOA world. But there is work to adapt WOA to the enterprise and while we’re seeing early successes with RIA models like Ajax, which are natural fits for a SOA and consume Web services naturally, there is more work to be done to figure out how governance, security, orchestration, and other issues will look like. Because like many things in the Web 2.0 world that are then adapted to business, there is usually an “enterprise context” that has to be added to make them workable inside our organizations.

What sort of things might we see in an organization that is trying to adopt WOA? While WOA adoption is actually happening at a grassroots level in many organizations already, we can use those early stories and what’s happening on the consumer Web to give us an idea:

What WOA looks like in the enterprise:

• A rich web of REST resources. Instead of a few point SOA services, enterprise data will be exposed through millions of granular REST resources (like the Web itself), which almost any application than can consume HTTP and XML can use. Much higher levels of syndication using RSS and ATOM will also be a hallmark of WOA adoption as it has been so successful in unleashing the Web of data on the Internet.
• Simple tools to weave the Web of resources into new applications.There are is a rapidly growing set of tools becoming available to build mashups quickly out of WOA (and SOA if you must) resources. Increasingly, these tools can be used by end-users but developers can very easily compose WOA resource into new solutions. IT developers will also use these tools as well as their own to create interoperability and integration where needed, like they do with SOA now.
• Highly consumable and reusable WOA “parts” including widgets, gadgets, and embedded social apps. There are now tens of thousands of Web components that can project data and functioanlity throughout the Web (and the enterprise) with a simple point and click. In combination with tools like blogs, wikis, and anything else which lets users add markup, this lets users build basic applications out of commonly available parts. Enterprise adopting WOA will take their own WOA resources and package them using popular “widget models” like Google’s OpenSocial and many others. The very presence of these Web parts drives viral adoption of a WOA as other users see them and use them, spreading them to the corners of the organization and beyond.
• Open Web APIs exposed on the Internet to ad hoc partners. Many organizations will start usintheir WOA to work with existing partners and start going into the Web services business, ala Amazon, themselves. We are learning that many organizations are highly underleveraging their vast assets in data and functionality and open them up to the Web in a dynamic way allow innovation and growth to be pushed out in the cloud and exciting new scenarios emerge. And as we’ve seen in many cases on the Web now, it’s also a direct revenue driver (see middle here).

The future is bright for SOA but it’s become clear that the SOA world will have to change some basic assumptions about the approaches that are used for design and implementation. Because the consensus is growing that the techniques riding under the aegis of WOA will be the ones to bet on.

Get a full overview of the intersection between Web 2.0 and the enterprise.

The WOA story emerges as better outcomes sought for SOA

http://blogs.zdnet.com/Hinchcliffe/?p=213

September 6th, 2008

Over the summer the enterprise IT blogosphere was swept up in a conversation around the concepts that many are calling Web-Oriented Architecture, or WOA. A different way to think about service-oriented architecture, WOA extolls a different but related set of technologies, in particular how to apply them in specific ways to connect our systems together into the solutions we need to take on our daily business challenges. WOA offers the exciting and fast-growth promise of the Web 2.0 world, while SOA has been seen as struggling and encountering low engagement in most organizations.

For those just joining the conversation, SOA is the most common set of top-level organizing principles and technologies that enterprises use to organize and connect their IT systems. However, SOA is increasingly in the firing line for less-than-stellar results and lack of business alignment. Few promising solutions for this have emerged lately, with the increasingly notable exception of WOA. WOA describes a compelling new focus that can address many existing SOA issues, but is sometimes at odds with traditional IT and business thinking.

Along with different technology emphasis, WOA offers a compelling new perspective on service uptake and consumption and offers potent ways of thinking about business models that can directly drive innovation and growth. Even better, we can now point to existing WOA success stories, albeit most of them in the online world. In short, SOA (of which WOA is a part) hasn’t looked this interesting in years. But like most new ideas, it inevitably faces challenges from the old guard.

For its own part, far from being a boring, back-office story about plumbing and infrastructure, SOA has actually seen better results than most of theenterprise architecture models that came before it. However, these returns have been fairly lackluster compared to what most business were actually looking for and what SOA practitioners wanted and were actively trying to achieve, certainly when any measurements of the ROI were taken. My detailed WOA overview last April tells the story: The Burton Group ultimately concluded earlier this year that “that SOA is not working in most organizations” based on extensive conversations with clients.

I’ve covered this territory a number of times in the past, most notably with an in-depth exploration of What is WOA?, but the story remains the same: WOA is being driven by the widespread success that lightweight Web services — and particularly their use in open APIs — are having on the open Web. The broad lesson that has been dawning on the enterprise architecture world this year is that this is what’s actually working in terms of what SOA has been trying to accomplish, but with a uniquely different approach.

Explore several WOA success stories and how they are driving SOA.

The tide seems to be turning in terms of the industry’s perspective of WOA as well. Respected SOA expert David Linthicum recently asked “SOA out, WOA in?” and seemed to think it was, noting it will take a long time, like SOA did, to make inroads in the enterprise despite its widespread adoption on the Internet. ZDNet’s own Joe McKendrick recently noted that “WOA wins hands-down over SOA in popularity contest” and Dave Rosenberg recentlydiscussed WOA on CNET and took it as a forgone conclusion. And this is a key point: Many organizations I talk to are already using some WOA to some degree on the ground today, it’s just not being promoted like traditional SOA is, thereby missing the benefit of the support, documentation, guidance, management, and infrastructure/tools support needed to fully flourish.

We have started to see traditional organizations begin to offer WOA-friendly services to the world at large. For example, the World Bank recently opened its Web API to developers using the increasingly popular Mashery service, which allows an organization to outsource their WOA. Of course, WOA can be used solely inside the firewall but some of the most interesting scenarios involve integration with business partners, on demand in a very agile, lightweight fashion.

And in the end, this is the challenge. The use of WOA on the technology side is only interesting if there is support for the business for the scenarios it encourages. You could convert all your Web services from SOAP or REST and be fully ready for the resulting stream of consumer and enterprise mashups, API customers, and hundreds of new business partners, but not if you’ve not redesigned your business a bit. This is also one key reason WOA isn’t synonymous with REST. WOA is architecture, both technical and business, while REST is a style building WOA services. The implications of WOA also go beyond REST to include other Web-oriented scenarios such as widgets, browser-based interfaces, and so on.

WOA entails both technology and business change

Unfortunately, many businesses have not yet absorbed the lessons of the Web 2.0 era and still look at the Web simply as a way to deliver Web pages. This limited view and understanding of the Web’s potential means that most organizations do not have it on the radar to link themselves together in the enterprise-wide and Web-wide ecosystems of creation and integration that WOA can enable. SOA has always been about connecting systems and people together and — at long last — we have a clear path to potentially wonderful outcomes in terms of unintended uses. This includes the ability to access business opportunities inside of time windows which would previously have been unattainable with our traditional, heavyweight SOA models. But only if we truly change the way we think about how to leverage the network.

One last thing, it’s important to remember that no small system can sustain contact with a large system for very long without being fundamentally changed by it. This is what is happening with businesses (the small system, no matter how large) and the Web today (the big system.) The intrinsic nature of the Web is driving major changes in how we create network-based products and services and is inexorably turning us into Web-oriented businesses. Businesses that want to be successful on this network without understanding its fundamental nature and capabilities are only delaying the time it takes to reach the full potential the Web offers.

In this way, WOA often describes network business models (such as open Web APIs) that often seem very foreign to non-Internet businesses but are powerfully aligned with the way that the Web works. These models are almost certainly essential to be successful and flourish in the modern competitive landscape on our networks today. In this way, too many organizations will ignore adding a WOA aspect to their SOA work until it’s too late and the ability to generate strong network effects in their industry is greatly reduced.

WOA is just one of a set of transformative new distribution models for network-based systems.

So how do organizations start down this route to investigate the WOA way of doing SOA and seeing if it works for them?

Like many aspects of Web 2.0, WOA is not complex or overly expensive, it’s a way of thinking about interacting over the network and all the classic SOA principles still apply, which just create and expose them differently.

1. Learn about WOA. Study the technology (HTTP, REST, syndication, open Web APIs, widgets, metadata documentation, Ajax, mashups, JSON, etc.), as well as the business and implementation side, including partner ecosystems, developer support sites, monetization, and chargebacks.
2. Adapt WOA to your organization. Every organization will have a landscape of existing SOA approaches and technologies that WOA approaches will need to be added to. Furthermore, WOA does little good unless you’re willing to use it for what it does well: Provide the fuel for RIA-powered portal applications, enterprise mashups, your public APIs, and so on. Begun working through how WOA security will work in your organization (inline or through HTTPS, for example) and other key starter issues that are (hopefully) already described in your SOA governance documents.
3. Conduct a pilot. Validate the items in #2 with a small pilot. Select amashup platform that works well for your organization and try it out. WOA enables SOA to be used in a much more agile, open, and effective manner, with the right tools involved but only in an environment that supports it all the way through the “stack” from browser, server, database, development tools, and management infrastructure.