The Role of a Data Services Layer in SOA

UK-trade journal, Database & Network Journal, has published one of my articles in their January issue: “Bringing Data into Service: The Role of a Data Services Layer in SOA”. I'm not aware of any Web version of it, I'm looking for a solution to make it available online. Stay tuned.

You can read the article here.

How MDM and Data Services can be Complementary

I'm pleased to inform you that DM Review has published my article about Data Services Platforms and Master Data Management.

In the past, I used to think that MDM was mostly an issue for EII products. Today, most EII have turned themselves into Data Services, but there are other approaches to Data Services Platforms. Based on my discussions my some MDM vendors I'm now convinced that DSP platforms have an important role to play in that space. You'll see all the details in the article.

SOA is dead?

First of all, I wish you all the best for 2009!

This new year starts with an obituary news, from Ann Thomas Mann (Burton Group): SOA is dead. This blog entry has generated a lot of reactions, and that was probably one of its aims:

• InfoQ - Debate Is SOA Dead
• ZDNet - SOA dead as of January 1st 2009
• Yahoo Tech - SOA Gets an obituary
• eWeek - SOA wanted dead or alive
• eBizq - So, if SOA is dead, should we focus more on the data?

(just to mention a few of them).

In fact, what she says is that service-orientation is still needed but the SOA TLA is now too heavily associated with the idea of project failure.

First comments:

• Seems to me all this is more provocation than anything else, done to generate some noise and visibility. Regarding this, it is a success.
• She is just writing what many were thinking and even saying since many years.
• People tend to quickly burn the idols they have created themselves. So that they can quickly idolize another one... with the same unreasonable passion.
• Sometimes it needs time, even for a good idea, to impose itself. Unfortunately, the World and the software industry are now stuck in short-term visions and immediate ROI, there is no more time for ideas to mature. New concepts now have to be perfect from inception, there is no room for improvement.
• If we agree that SOA is not a technology but rather a way of thinking software architectures, then saying that "SOA is dead but there is still a need for the idea of services" is a kind of oxymore.
• If SOA is really wrong, changing it's name won't fix it. WOA, RESTFul, Mashups won't save services if the main idea behind it is not good. The problem cannot just be with the SOA name.

So, what was really wrong with this supposedly dying SOA?

1. SOA is much more complicated than expected. Remember: the 'S' in SOAP means "simple". SOA was supposed to be simpler than CORBA and J2E. But why SOA would have been less complicated than its predecessors? When you want to design non-trivial business applications by assembling distributed components you will face complexity. Using HTTP / SOAP instead of IIOP and WSDL instead of IDL will not fundamentally change the thing. The problem is still complex and unfortunately (or not) we don't have easy solutions to complex problems yet. Thinking large systems as communicating distributed agents is not so easy. It is all about parallel asynchronous programming. Most developers are not trained for that, as they were not trained to be Java gurus. It is so funny to see that today SOA is attacked on the simplicity front.
2. SOA is too much process-centric. And once again, it was the same with CORBA and J2E. All distributed arhcitectures are first trying to solve the easy problem first: distribute processes. Then they start to deal with data and the real problems raise.

But it is not necessarily too late for SOA, as far as we can reconcile data and processes, states and behaviors, attributes and methods, however you name it.

So let's go back to the fundamentals. David Linthicum is thinking in the right direction: How to deal with data in SOA?

That's where we are today in SOA. Data Services are clearly going in the right direction. Customers will be able to solve some of their data integration issues with SOA, and they will have time to understand the value of this approach. To some extent, Data Services are at the cross between the promises of SOA and OO: states encapsulated by business methods.

Obviously, we'll need to go further, move from simple data federation to adaptative mediation, but we'll have time as we are solving real problems in the meantime.

Data Services should not be too much focused on names like SOA, WOA, RESTFul, mashups. New architectures are not replacing old ones (even if their proponents always tend to think it, at the beginning). They just complement them. Data Services platforms should equally support all these styles. Data Services Platforms should publish integrated data as SOA-or-not services, but also as POJOs, xDBC... The same way they have to connect to SOA-or-not data services as data sources, but also to RDBMS, packaged applications, mainframe transactions and screens, etc. The real IT world will certainly never be all-SOA, but equally it won't be all-non-SOA.

Services are basically components. There are many deep reasons why imposing a component-oriented design has continuously failed for decades. SOA is a new try and it won't be the last one. Component-based progamming models have failed for cultural reasons (programmers are not ready for that) and also because of the Lego model. You can't build a complex system by manually linking thousands of atomic components. SOA is way too much manually hard-coded today. We need to be able to dynamically generate, publish and compose services at runtime. We need to design new data access patterns, maybe based on asynchronous, reactive models. This will require advanced metadata, more semantic metadata and dynamic computation of chains of service calls at runtime. All this won't be achieved in few years.

In the meantime, programmers will progressively have to accept they won't control the flow of service calls, exactly as DBAs had to accept they won't tune all SQL statements any longer with the raise of persistence technologies. Each time you add abstraction layers you are able to do more, but conversely you lose some visibility and control on the lower layers.

So, the real debate is not what's next after SOA?, or which style of service-orientation will win? REST is not simpler than SOA if you want to manage complex things by manually connecting services. If one day we want to solve complex problems with simple solutions, then we'll need new abstraction layers, we need dynamic SOA. And for that, we need to stop manually calling and connecting services. Regarding this SOA versus WOA or SOAP versus REST is not the point at all. Services are a required piece in the puzzle, but we need new abstractions on top of them.

See the funny REST is dead blog entry here.