This chapter is from the book
This chapter is from the book
This chapter is from the book
Messaging
An enterprise has multiple applications that are being built independently, with different languages and platforms. The enterprise needs to share data and processes in a responsive way.
How can I integrate multiple applications so that they work together and can exchange information?
How can I integrate multiple applications so that they work together and can exchange information?
File Transfer and Shared Database enable applications to share their data but not their functionality. Remote Procedure Invocation enables applications to share functionality, but it tightly couples them as well. Often the challenge of integration is about making collaboration between separate systems as timely as possible, without coupling systems together in such a way that they become unreliable either in terms of application execution or application development.
File Transfer allows you to keep the applications well decoupled but at the cost of timeliness. Systems just can't keep up with each other. Collaborative behavior is way too slow. Shared Database keeps data together in a responsive way but at the cost of coupling everything to the database. It also fails to handle collaborative behavior.
Faced with these problems, Remote Procedure Invocation seems an appealing choice. But extending a single application model to application integration dredges up plenty of other weaknesses. These weaknesses start with the essential problems of distributed development. Despite that RPCs look like local calls, they don't behave the same way. Remote calls are slower, and they are much more likely to fail. With multiple applications communicating across an enterprise, you don't want one application's failure to bring down all of the other applications. Also, you don't want to design a system assuming that calls are fast, and you don't want each application knowing the details about other applications, even if it's only details about their interfaces.
What we need is something like File Transfer in which lots of little data packets can be produced quickly and transferred easily, and the receiver application is automatically notified when a new packet is available for consumption. The transfer needs a retry mechanism to make sure it succeeds. The details of any disk structure or database for storing the data needs to be hidden from the applications so that, unlike Shared Database, the storage schema and details can be easily changed to reflect the changing needs of the enterprise. One application should be able to send a packet of data to another application to invoke behavior in the other application, like Remote Procedure Invocation, but without being prone to failure. The data transfer should be asynchronous so that the sender does not need to wait on the receiver, especially when retry is necessary.
Use Messaging to transfer packets of data frequently, immediately, reliably, and asynchronously, using customizable formats.
)
Use Messaging to transfer packets of data frequently, immediately, reliably, and asynchronously, using customizable formats.
Asynchronous messaging is fundamentally a pragmatic reaction to the problems of distributed systems. Sending a message does not require both systems to be up and ready at the same time. Furthermore, thinking about the communication in an asynchronous manner forces developers to recognize that working with a remote application is slower, which encourages design of components with high cohesion (lots of work locally) and low adhesion (selective work remotely).
Messaging systems also allow much of the decoupling you get when using File Transfer. Messages can be transformed in transit without either the sender or receiver knowing about the transformation. The decoupling allows integrators to choose between broadcasting messages to multiple receivers, routing a message to one of many receivers, or other topologies. This separates integration decisions from the development of the applications. Since human issues tend to separate application development from application integration, this approach works with human nature rather than against it.
The transformation means that separate applications can have quite different conceptual models. Of course, this means that semantic dissonance will occur. However, the messaging viewpoint is that the measures used by Shared Database to avoid semantic dissonance are too complicated to work in practice. Also, semantic dissonance is going to occur with third-party applications and with applications added as part of a corporate merger, so the messaging approach is to address the issue rather than design applications to avoid it.
By sending small messages frequently, you also allow applications to collaborate behaviorally as well as share data. If a process needs to be launched once an insurance claim is received, it can be done immediately by sending a message when a single claim comes in. Information can be requested and a reply made rapidly. While such collaboration isn't going to be as fast as Remote Procedure Invocation, the caller needn't stop while the message is being processed and the response returned. And messaging isn't as slow as many people think—many messaging solutions originated in the financial services industry where thousands of stock quotes or trades have to pass through a messaging system every second.
This book is about Messaging, so you can safely assume that we consider Messaging to be generally the best approach to enterprise application integration. You should not assume, however, that it is free of problems. The high frequency of messages in Messaging reduces many of the inconsistency problems that bedevil File Transfer, but it doesn't remove them entirely. There are still going to be some lag problems with systems not being updated quite simultaneously. Asynchronous design is not the way most software people are taught, and as a result there are many different rules and techniques in place. The messaging context makes this a bit easier than programming in an asynchronous application environment like X Windows, but asynchrony still has a learning curve. Testing and debugging are also harder in this environment.
The ability to transform messages has the nice benefit of allowing applications to be much more decoupled from each other than in Remote Procedure Invocation. But this independence does mean that integrators are often left with writing a lot of messy glue code to fit everything together.
Once you decide that you want to use Messaging for system integration, there are a number of new issues to consider and practices you can employ.
-
How do you transfer packets of data?
A sender sends data to a receiver by sending a Message via a Message Channel that connects the sender and receiver.
-
How do you know where to send the data?
If the sender does not know where to address the data, it can send the data to a Message Router, which will direct the data to the proper receiver.
-
How do you know what data format to use?
If the sender and receiver do not agree on the data format, the sender can direct the data to a Message Translator that will convert the data to the receiver's format and then forward the data to the receiver.
-
If you're an application developer, how do you connect your application to the messaging system?
An application that wishes to use messaging will implement Message Endpoints to perform the actual sending and receiving.