Enterprise Integration Patterns: Messaging Channels

This chapter is from the book 

Introduction

In Chapter 3, “Messaging Systems,” we discussed Message Channels (60). When two applications want to exchange data, they do so by sending the data through a channel that connects the two. The application sending the data may not know which application will receive the data. However, by selecting a particular channel on which to send the data, the sender knows that the receiver will be one that is looking for that sort of data by looking for it on that channel. In this way, the applications that produce shared data have a way to communicate with those that wish to consume it.

Message Channel Themes

Deciding to use a Message Channel (60) is the simple part; if an application has data to transmit or data it wishes to receive, it will have to use a channel. The challenge is knowing what channels your applications will need and what to use them for.

• Fixed set of channels—One topic discussed in this chapter is that the set of Message Channels (60) available to an application tends to be static. When designing an application, a developer must know where to put what types of data in order to share that data with other applications and likewise where to look for particular types of data coming from other applications. These paths of communication cannot be dynamically created and discovered at runtime; they need to be agreed upon at design time so that the application knows where its data is coming from and where the data is going to. (While it is true that most channels must be statically defined, there are exceptions, cases where dynamic channels are practical and useful. One exception is the reply channel in Request-Reply (154). The requestor can create or obtain a new channel that the replier knows nothing about and specify it as the Return Address (159) of a request message. The replier can then use it. Another exception is messaging system implementations that support hierarchical channels. A receiver can subscribe to a parent in the hierarchy, and then a sender can publish to a new child channel that the receiver knows nothing about. The subscriber will still receive the message. These relatively unusual cases notwithstanding, channels are usually defined before deployment, and applications are designed around a known set of channels.)
• Determining the set of channels—A related issue is, Who decides what Message Channels (60) are available—the messaging system or the applications? In other words, does the messaging system define certain channels and require the applications to make do with those? Or do the applications determine what channels they need and require the messaging system to provide them? There is no simple answer; designing the needed set of channels is iterative. First, the applications determine which channels the messaging system needs to provide. Subsequent applications will try to design their communication around the channels that are available, but when this is not practical, they will require that additional channels be added. When a set of applications already uses a certain set of channels, and new applications want to join in, they too will use the existing set of channels. When existing applications add new functionality, they may require new channels.
• Unidirectional channels—Another common source of confusion is whether a Message Channel (60) is unidirectional or bidirectional. Technically, it’s neither; a channel is more like a bucket that some applications add data to and other applications take data from (albeit a bucket that is distributed across multiple computers in some coordinated fashion). But because the data is in messages that travel from one application to another, that gives the channel direction, making it unidirectional. If a channel were bidirectional, that would mean that an application would both send messages to and receive messages from the same channel, which—while technically possible—makes little sense because the application would tend to keep consuming its own messages: the messages it’s supposed to be sending to other applications. So, for all practical purposes, channels are unidirectional. As a consequence, for two applications to have a two-way conversation, they need two channels, one in each direction (see Request-Reply [154] in the next chapter).

Message Channel Decisions

Now that we understand what Message Channels (60) are, let’s consider the decisions involved in using them.

• One-to-one or one-to-many—When your application shares a piece of data, do you want to share it with just one other application or with any other application that is interested? To send the data to a single application, use a Point-to-Point Channel (103). This does not guarantee that every piece of data sent on that channel will necessarily go to the same receiver, because the channel might have multiple receivers. It does, however, ensure that any one piece of data will be received by only one of the applications. If you want all of the receiver applications to be able to receive the data, use a Publish-Subscribe Channel (106). When you send a piece of data this way, the channel effectively copies the data for each of the receivers.
• What type of data—Any data in any computer memory has to conform to some sort of type: a known format or expected structure with an agreed-upon meaning. Otherwise, all data would just be a bunch of bytes, and there would be no way to make any sense out of it. Messaging systems work much the same way; the message contents must conform to some type so that the receiver understands the data’s structure. Datatype Channel (111) is the idea that all of the data on a channel must be of the same type. This is the main reason that messaging systems need lots of channels. If the data could be of any type, the messaging system would only need one channel (in each direction) between any two applications.
• Invalid and dead messages—The message system can ensure that a message is delivered properly, but it cannot guarantee that the receiver will know what to do with it. The receiver has expectations about the data’s type and meaning. When it receives a message that doesn’t meet these expectations, there’s not much it can do. What it can do, though, is put the strange message on a specially designated Invalid Message Channel (60) in hopes that some utility monitoring the channel will pick up the message and figure out what to do with it. Many messaging systems have a similar built-in feature, a Dead Letter Channel (119), for messages that are successfully sent but ultimately cannot be successfully delivered. Again, a system management utility should monitor the Dead Letter Channel (119) and decide what to do with the messages that could not be delivered.
• Crash proof—If the messaging system crashes or is shut down for maintenance, what happens to its messages? When it is back up and running, will its messages still be in its channels? By default, no; channels store their messages in memory. However, Guaranteed Delivery (122) makes channels persistent so that their messages are stored on disk. This hurts performance but makes messaging more reliable, even when the messaging system isn’t.
• Non-messaging clients—What if an application cannot connect to a messaging system but still wants to participate in messaging? Normally it would be out of luck, but if the messaging system can connect to the application somehow—through its user interface, its business services API, its database, or a network connection such as TCP/IP or HTTP—then a Channel Adapter (127) on the messaging system can be used. This allows you to connect a channel (or set of channels) to the application without having to modify the application and perhaps without requiring a messaging client running on the same machine as the application. Sometimes the “non-messaging client” really is a messaging client, but just for a different messaging system. In that case, an application that is a client on both messaging systems can build a Messaging Bridge (133) between the two, effectively connecting them into one composite messaging system.
• Communications backbone—As more and more of an enterprise’s applications connect to the messaging system and make their functionality available through messaging, the messaging system becomes a centralized point of one-stop shopping for shared functionality in the enterprise. A new application simply needs to know which channels to use to request functionality and which others to listen on for the results. The messaging system itself essentially becomes a Message Bus (137), a backbone providing access to all of the enterprise’s various and ever-changing applications and functionality. You can achieve this integration nirvana more quickly and easily by specifically designing for it from the beginning.

As you can see, getting applications set up for Messaging (53) involves more than just connecting them to the messaging system so that they can send messages. The messages must have Message Channels (60) to transmit on. Simply slapping in some channels doesn’t get the job done either. They have to be designed with a purpose, based on the datatype being shared, the sort of application making the data available, and the sort of application receiving the data. This chapter explains the decisions that go into designing these channels.

To help illustrate the patterns, each one has an example from a fictitious, simplified stock trading domain. While none of these examples should be used as the basis for implementing a real trading system, they do serve as brief and specific examples of how the patterns can be used.

Point-to-Point Channel

An application is using Messaging (53) to make remote procedure calls (RPCs) or transfer documents.

How can the caller be sure that exactly one receiver will receive the document or perform the call?

One advantage of an RPC is that it’s invoked on a single remote process, so either that receiver performs the procedure or it does not (and an exception occurs). And since the receiver was called only once, it performs the procedure only once. But with messaging, once a call is packaged as a Message (66) and placed on a Message Channel (60), potentially many receivers could see it on the channel and decide to perform the procedure.

The messaging system could prevent more than one receiver from monitoring a single channel, but this would unnecessarily limit callers that wish to transmit data to multiple receivers. All of the receivers on a channel could coordinate to ensure that only one of them actually performs the procedure, but that would be complex, create a lot of communications overhead, and generally increase the coupling between otherwise independent receivers. Multiple receivers on a single channel may be desirable so that multiple messages can be consumed concurrently, but any one receiver should consume any single message.

Send the message on a Point-to-Point Channel, which ensures that only one receiver will receive a particular message.

A Point-to-Point Channel ensures that only one receiver consumes any given message. The channel can have multiple receivers that can consume multiple messages concurrently, but only one of them can successfully consume a particular message. If multiple receivers try to consume a single message, the channel ensures that only one of them succeeds, so the receivers do not have to coordinate with each other.

When a Point-to-Point Channel has only one consumer, the fact that a message gets consumed only once is not surprising. When the channel has multiple consumers, then they become Competing Consumers (502), and the channel ensures that only one of the consumers receives each message. This design makes consuming and processing messages highly scalable because that work can be load-balanced across multiple consumers running in multiple applications on multiple computers.

Whereas you use a Point-to-Point Channel to send a message to only one of the available receivers, to send a message to all available receivers, use a Publish-Subscribe Channel (106). To implement RPCs using messaging, use Request-Reply (154) with a pair of Point-to-Point Channels. The call is a Command Message (145), and the reply is a Document Message (147).

---

Example: Stock Trading

In a stock trading system, the request to make a particular trade is a message that should be consumed and performed by exactly one receiver, so the message should be placed on a Point-to-Point Channel.

---

---

Example: JMS Queue

In JMS, a point-to-point channel implements the Queue interface. The sender uses a QueueSender to send messages; each receiver uses its own QueueReceiver to receive messages [JMS 1.1], [Hapner].

An application uses a QueueSender to send a message like this:

Queue queue = // obtain the queue via JNDI 
QueueConnectionFactory factory = // obtain the connection factory via JNDI 
QueueConnection connection = factory.createQueueConnection(); 
QueueSession session = connection.createQueueSession(true, Session.AUTO_ACKNOWLEDGE); 
QueueSender sender = session.createSender(queue);  

Message message = session.createTextMessage("The contents of the message.");  

sender.send(message);

An application uses a QueueReceiver to receive a message like this:

Queue queue = // obtain the queue via JNDI 
QueueConnectionFactory factory = // obtain the connection factory via JNDI 
QueueConnection connection = factory.createQueueConnection(); 
QueueSession session = connection.createQueueSession(true, Session.AUTO_ACKNOWLEDGE); 
QueueReceiver receiver = session.createReceiver(queue);  

TextMessage message = (TextMessage) receiver.receive(); 
String contents = message.getText();

---

---

Example: .NET MessageQueue

In .NET, the MessageQueue class implements a point-to-point channel [SysMsg]. MSMQ, which implements .NET messaging, supported only point-to-point messaging prior to version 3.0, so point-to-point is what .NET supports. Whereas JMS separates the responsibilities of the connection factory, connection, session, sender, and queue, a MessageQueue does it all.

Send a message on a MessageQueue like this:

MessageQueue queue = new MessageQueue("MyQueue"); 
queue.Send("The contents of the message.");

Receive a message on a MessageQueue like this:

MessageQueue queue = new MessageQueue("MyQueue"); 
Message message = queue.Receive(); 
String contents = (String) message.Body();

---

Publish-Subscribe Channel

An application is using Messaging (53) to announce events.

How can the sender broadcast an event to all interested receivers?

Luckily, there are well-established patterns for implementing broadcasting. The Observer pattern [GoF] describes the need to decouple observers from their subject (that is, the originator of the event) so that the subject can easily provide event notification to all interested observers no matter how many observers there are (even none). The Publisher-Subscriber pattern [POSA] expands upon Observer by adding the notion of an event channel for communicating event notifications.

That’s the theory, but how does it work with messaging? The event can be packaged as a Message (66) so that messaging will reliably communicate the event to the observers (subscribers). Then, the event channel is a Message Channel (60). But how will a messaging channel properly communicate the event to all of the subscribers?

Each subscriber needs to be notified of a particular event once but should not be notified repeatedly of the same event. The event cannot be considered consumed until all of the subscribers have been notified, but once they have, the event can be considered consumed and should disappear from the channel. Yet, having the subscribers coordinate to determine when a message is consumed violates the decoupling of the Observer pattern. Concurrent consumers should not compete but should be able to share the event message.

Send the event on a Publish-Subscribe Channel, which delivers a copy of a particular event to each receiver.

A Publish-Subscribe Channel works like this: It has one input channel that splits into multiple output channels, one for each subscriber. When an event is published into the channel, the Publish-Subscribe Channel delivers a copy of the message to each of the output channels. Each output end of the channel has only one subscriber, which is allowed to consume a message only once. In this way, each subscriber gets the message only once, and consumed copies disappear from their channels.

A Publish-Subscribe Channel can be a useful debugging tool. Even though a message is destined to only a single receiver, using a Publish-Subscribe Channel allows you to eavesdrop on a message channel without disturbing the existing message flow. Monitoring all traffic on a channel can be tremendously helpful when debugging messaging applications. It can also save you from inserting a ton of print statements into each application that participates in the messaging solution. Creating a program that listens for messages on all active channels and logs them to a file can provide many of the same benefits that a Message Store (555) brings.

However, the ability to eavesdrop on a Publish-Subscribe Channel can also turn into a disadvantage. If your messaging solution transmits payroll data between the payroll system and the accounting system, you may not want to allow anyone to write a simple program to listen to the message traffic. Point-to-Point Channels alleviate the problem somewhat: Because the eavesdropper would consume messages off the channel and messages would suddenly be missing, the situation could be detected very quickly. However, many message queue implementations provide peek functions that let consumers look at messages inside a queue without consuming any of the messages. As a result, subscribing to a Message Channel (60) is an operation that should be restricted by security policies. Many (but not all) commercial messaging implementations implement such restrictions. In addition, creating a monitoring tool that logs active subscribers to Message Channels (60) can be a useful systems management tool.

An Event Message (151) is usually sent on a Publish-Subscribe Channel because multiple dependents are often interested in an event. A subscriber can be durable or nondurable—see Durable Subscriber (522) in the Chapter 10, “Messaging Endpoints.” If notifications should be acknowledged by the subscribers, use Request-Reply (154), where the notification is the request and the acknowledgment is the reply. Storing each message on a Publish-Subscribe Channel until all subscribers consume the message can require a large amount of message storage. To help alleviate this issue, messages sent to a Publish-Subscribe Channel can use Message Expiration (176).

---

Example: Stock Trading

In a stock trading system, many systems may need to be notified of the completion of a trade, so make them all subscribers of a Publish-Subscribe Channel that publishes trade completions.

Likewise, many consumers are interested in displaying or processing current stock quote data. Therefore, stock quotes should be broadcast across a Publish-Subscribe Channel.

---

---

Example: JMS Topic

In JMS, a Publish-Subscribe Channel implements the Topic interface. The sender uses a TopicPublisher to send messages; each receiver uses its own Topic Subscriber to receive messages [JMS 1.1], [Hapner].

An application uses a TopicPublisher to send a message like this:

Topic topic = // obtain the topic via JNDI 
TopicConnectionFactory factory = // obtain the connection factory via JNDI 
TopicConnection connection = factory.createTopicConnection(); 
TopicSession session = connection.createTopicSession(true, Session.AUTO_ACKNOWLEDGE); 
TopicPublisher publisher = session.createPublisher(topic);  

Message message = session.createTextMessage("The contents of the message.");  

publisher.publish(message);

An application uses a TopicSubscriber to receive a message like this:

Topic topic = // obtain the topic via JNDI 
TopicConnectionFactory factory = // obtain the connection factory via JNDI 
TopicConnection connection = factory.createTopicConnection(); 
TopicSession session = connection.createTopicSession(true, Session.AUTO_ACKNOWLEDGE); 
TopicSubscriber subscriber = session.createSubscriber(topic);  

TextMessage message = (TextMessage) subscriber.receive(); 
String contents = message.getText();

---

---

Example: MSMQ One-to-Many Messaging

A new feature in MSMQ 3.0 [MSMQ] is a one-to-many messaging model, which has two different approaches.

1. Real-Time Messaging Multicast—This approach most closely matches publish-subscribe, but its implementation is entirely dependent on IP multicasting via the Pragmatic General Multicast (PGM) protocol, so this feature cannot be used with other protocols that are not based on IP.
2. Distribution Lists and Multiple-Element Format Names—A Distribution List enables the sender to explicitly send a message to a list of receivers (but this violates the spirit of the Observer pattern because the sender now has to be aware of the receivers). Therefore, this feature more closely resembles a Recipient List (249) than a Publish-Subscribe Channel. A Multiple-Element Format Name is a symbolic channel specifier that dynamically maps to multiple real channels, which is more the spirit of a Publish-Subscribe Channel but still forces the sender to choose between a real channel and a symbolic one.

The .NET Common Language Runtime (CLR) does not provide direct support for using the one-to-many messaging model. However, this functionality can be accessed through the COM interface [MDMSG], which can be embedded in .NET code.

---

---

Example: Simple Messaging

The JMS Publish-Subscribe example in Chapter 6, “Interlude: Simple Messaging,” shows an example of how to implement Observer across multiple processes using messaging.

---

Datatype Channel

An application is using Messaging (53) to transfer different types of data, such as different types of documents.

How can the application send a data item such that the receiver will know how to process it?

All messages are just instances of the same message type, as defined by the messaging system, and the contents of any message are ultimately just a byte array. While this simple structure—a bundle of bytes—is specific enough for a messaging system to be able to transmit a message, it is not specific enough for a receiver to be able to process a message’s contents.

A receiver must know the message content’s data structure and data format. The structure could be character array, byte array, serialized object, XML document, and so on. The format could be the record structure of the bytes or characters, the class of the serialized object, the schema definition of the XML document, and so on. All of this knowledge is loosely referred to as the message’s type, referring to both the structure and format of the message’s contents.

The receiver must know what type of messages it’s receiving, or it won’t know how to process them. For example, a sender may wish to send different objects such as purchase orders, price quotes, and queries. Yet, a receiver will probably take different steps to process each of those, so it has to know which is which. If the sender simply sends all of these to the receiver via a single message channel, the receiver will not know how to process each one.

The sender knows what message type it’s sending, so how can this be communicated to the receiver? The sender could put a flag in the message’s header (see Format Indicator [180]), but then the receiver will need a case statement. The sender could wrap the data in a Command Message (145) with a different command for each type of data, but that presumes to tell the receiver what to do with the data when all that the message is trying to do is transmit the data to the receiver.

A similar problem—one separate from messaging—occurs when processing a collection of items. The collection must be homogeneous, meaning that all of the items are the same type, so that the processor knows what an item’s type is and thus how to manipulate it. Many collection implementations do not force all of the items to be of a specific type, so programmers must design their code to ensure that all of the items in a collection are of the same type. Otherwise, items of different types can be added to a collection, but the code processing those items will not know how to manipulate each one because it won’t know what type a particular item implements.

The same principle applies to messaging because the messages on a channel must likewise all be of the same type. The simplest solution is for all of the messages to be of the same format. If the formats must differ, they must all have a reliable Format Indicator (180). While the channel does not force all messages to be of the same type, the receiver needs them to be so that it knows how to process them.

Use a separate Datatype Channel for each datatype so that all data on a particular channel is of the same type.

By using a separate Datatype Channel for each type of data, all of the messages on a given channel will contain the same type of data. The sender, knowing what type the data is, will need to select the appropriate channel to send it on. The receiver, knowing what channel the data was received on, will know what type it is.

As shown in the figures, since the sender wants to send three different types of data (purchase orders, price quotes, and queries), it should use three different channels. When sending an item, the sender must select the appropriate Datatype Channel for that item. When receiving an item, the receiver knows the item’s type because of the datatype channel on which it received the item.

As discussed under Message Channel (60), channels are cheap, but they are not free. An application may need to transmit many different datatypes, too many to create a separate Datatype Channel for each. In this case, multiple datatypes can share a single channel by using a different Selective Consumer (515) for each type. This makes a single physical channel act like multiple logical Datatype Channels (a strategy called multiplexing). Whereas Datatype Channel explains why all messages on a channel must be of the same format, Canonical Data Model (355) explains how all messages on all channels in an enterprise should follow a unified data model.

If we would like to use Datatype Channels but an existing message publisher simply sends all messages to a single channel, we can use a Content-Based Router (230) to demultiplex the messages. The router divides the message stream across multiple Datatype Channels, each of which carries messages of only one type.

A Message Dispatcher (508), besides providing concurrent message consumption, can be used to process a generic set of messages in type-specific ways. Each message must specify its type (typically by a format indicator in the message’s header); the dispatcher detects the message’s type and dispatches it to a type-specific performer for processing. The messages on the channel are still all of the same type, but that type is the more general one that the dispatcher supports, not the more specific ones that the various performers require.

A Format Indicator (180) is used to distinguish different format versions of the same data, which in turn enables these different formats to be sent on the same Datatype Channel.

---

Example: Stock Trading

In a stock trading system, if the format of a quote request is different from that of a trade request, the system should use a separate Datatype Channel for communicating each kind of request. Likewise, a change-of-address announcement may have a different format from a change-of-portfolio-manager announcement, so each kind of announcement should have its own Datatype Channel.

---

Invalid Message Channel

An application is using Messaging (53) to receive Messages (66).

How can a messaging receiver gracefully handle receiving a message that makes no sense?

In theory, everything on a Message Channel (60) is just a message, and message receivers just process messages. However, to process a message, a receiver must be able to interpret its data and understand its meaning. This is not always possible: The message body may cause parsing errors, lexical errors, or validation errors. The message header may be missing needed properties, or the property values may not make sense. A sender might put a perfectly good message on the wrong channel, transmitting it to the wrong receiver. A malicious sender could purposely send an incorrect message just to mess up the receiver. A receiver may not be able to process all the messages it receives, so it must have some other way to handle messages it does not consider valid.

A Message Channel (60) should be a Datatype Channel (111), where each of the messages on the channel is supposed to be of the proper datatype for that channel. If a sender puts a message on the channel that is not the correct datatype, the messaging system will transmit the message successfully, but the receiver will not recognize the message and will not know how to process it.

One example of a message with an improper datatype or format is a byte message on a channel that is supposed to contain text messages. Another example is a message whose format is not correct, such as an XML document that is not well formed or that is not valid for the agreed-upon DTD or schema. There’s nothing wrong with these messages as far as the messaging system is concerned, but the receiver will not be able to process them, so they are invalid.

Messages that do not contain the header field values that the receiver expects are also invalid. If a message is supposed to have header properties such as a Correlation Identifier (163), Message Sequence (170) identifiers, a Return Address (159), and so on, but the message is missing the properties, then the messaging system will deliver the message properly, but the receiver will not be able to process it successfully.

When the receiver discovers that the message it’s trying to process is not valid, what should it do with the message? It could put the message back on the channel, but then the message will just be reconsumed by the same receiver or another like it. Meanwhile, invalid messages that are being ignored will clutter the channel and hurt performance. The receiver could consume the invalid message and throw it away, but that would tend to hide messaging problems that need to be detected. What the system needs is a way to clean improper messages out of channels and put them in a place where they will be out of the way but can be detected to diagnose problems with the messaging system.

The receiver should move the improper message to an Invalid Message Channel, a special channel for messages that could not be processed by their receivers.

When designing a messaging system for applications to use, the administrator must define one or more Invalid Message Channels for the applications to use. The Invalid Message Channel will not be used for normal, successful communication, so if it is cluttered with improper messages, that will not be a problem. An error handler that wants to diagnose improper messages can use a receiver on the invalid channel to detect messages as they become available.

An Invalid Message Channel is like an error log for messaging. When something goes wrong in an application, it’s a good idea to log the error. When something goes wrong processing a message, it’s a good idea to put the message on the channel for invalid messages. If it won’t be obvious to anyone browsing the channel why this message is invalid, the application should also log an error with more details.

Keep in mind that a message is neither inherently valid nor invalid, but it is the receiver’s context and expectations that make this determination. A message that may be valid for one receiver may be invalid for another receiver; two such receivers should not share the same channel. A message that is valid for one receiver on a channel should be valid for all other receivers on that channel. Likewise, if one receiver considers a message invalid, all other receivers should as well. It is the sender’s responsibility to make sure that a message it sends on a channel will be considered valid by the channel’s receivers. Otherwise, the receivers will ignore the sender’s messages by rerouting them to the Invalid Message Channel.

A similar but separate problem occurs when a message is structured properly, but its contents are semantically incorrect. For example, a Command Message (145) may instruct the receiver to delete a database record that does not exist. This is not a messaging error but an application error. As such, while it may be tempting to move the message to the Invalid Message Channel, there is nothing wrong with the message, so treating it as invalid is misleading. Rather, an error like this should be handled as an invalid application request, not an invalid message.

This difference between message-processing errors and application errors becomes simpler and clearer when the receiver is implemented as a Service Activator (532) or Messaging Gateway (468). These patterns separate message-processing code from the rest of the application. If an error occurs while processing the message, the message is invalid and should be moved to the Invalid Message Channel. If it occurs while the application processes the data from the message, that is an application error that has nothing to do with messaging.

An Invalid Message Channel whose contents are ignored is about as useful as an error log that is ignored. Messages on the Invalid Message Channel indicate application integration problems, so those messages should not be ignored; rather, they should be analyzed to determine what went wrong so that the problem can be fixed. Ideally, this would be an automated process that consumed invalid messages, determined their cause, and fixed the underlying problems. However, the cause is often a coding or configuration error that requires a developer or system analyst to evaluate and repair. At the very least, applications that use messaging and Invalid Message Channels should have a process that monitors the Invalid Message Channel and alerts system administrators whenever the channel contains messages.

A similar concept implemented by many messaging systems is a Dead Letter Channel (119). Whereas an Invalid Message Channel is for messages that can be delivered and received but not processed, a Dead Letter Channel (119) is for messages that the messaging system cannot deliver properly.

---

Example: Stock Trading

In a stock trading system, an application for executing trade requests might receive a request for a current price quote, or a trade request that does not specify what security to buy or how many shares, or a trade request that does not specify to whom to send the trade confirmation. In any of these cases, the application has received an invalid message—one that does not meet the minimum requirements necessary for the application to be able to process the trade request. Once the application determines the message to be invalid, it should resend the message onto the Invalid Message Channel. The various applications that send trade requests may wish to monitor the Invalid Message Channel to determine if their requests are being discarded.

---

---

Example: JMS Specification

In JMS, the specification suggests that if a MessageListener gets a message it cannot process, a well-behaved listener should divert the message “to some form of application-specific ‘unprocessable message’ destination” [JMS 1.1]. This unprocessable message destination is an Invalid Message Channel.

---

---

Example: Simple Messaging

The JMS Request-Reply example and .NET Request-Reply example (both in Chapter 6, “Interlude: Simple Messaging”) show an example of how to implement receivers that reroute messages they cannot process to an Invalid Message Channel.

---

Dead Letter Channel

An enterprise is using Messaging (53) to integrate applications.

What will the messaging system do with a message it cannot deliver?

If a receiver receives a message it cannot process, it should move the invalid message to an Invalid Message Channel (60). But what if the messaging system cannot deliver the message to the receiver in the first place?

There are a number of reasons the messaging system may not be able to deliver a message. The messaging system may not have the message’s channel configured properly. The message’s channel may be deleted after the message is sent but before it can be delivered or while it is waiting to be received. The message may expire before it can be delivered (see Message Expiration [176]). A message without an explicit expiration may nevertheless time out if it cannot be delivered for a very long time. A message with a selection value that all Selective Consumers (515) ignore will never be read and may eventually die. A message could have something wrong with its header that prevents it from being delivered successfully.

Once the messaging system determines that it cannot deliver a message, it has to do something with the message. It could just leave the message wherever it is, cluttering up the system. It could try to return the message to the sender, but the sender is not a receiver and cannot detect deliveries. It could just delete the message and hope no one misses it, but this may well cause a problem for the sender that has successfully sent the message and expects it to be delivered (and received and processed).

When a messaging system determines that it cannot or should not deliver a message, it may elect to move the message to a Dead Letter Channel.

The specific way a Dead Letter Channel works depends on the specific messaging system’s implementation, if it provides one at all. The channel may be called a “dead message queue” [Monson-Haefel] or “dead letter queue” [MQSeries], [Dickman]. Typically, each machine the messaging system is installed on has its own local Dead Letter Channel so that whatever machine a message dies on, it can be moved from one local queue to another without any networking uncertainties. This also records what machine the message died on. When the messaging system moves the message, it may also record the original channel on which the message was supposed to be delivered.

The difference between a dead message and an invalid one is that the messaging system cannot successfully deliver what it then deems a dead message, whereas an invalid message is properly delivered but cannot be processed by the receiver. Determining if a message should be moved to the Dead Letter Channel is an evaluation of the message’s header performed by the messaging system. On the other hand, the receiver moves a message to an Invalid Message Channel (60) because of the message’s body or particular header fields the receiver is interested in. To the receiver, determination and handling of dead messages seem automatic, whereas the receiver must handle invalid messages itself. A developer using a messaging system is stuck with whatever dead message handling the messaging system provides, but she can design her own invalid message handling, including handling for seemingly dead messages that the messaging system doesn’t handle.

---

Example: Stock Trading

In a stock trading system, an application that wishes to perform a trade can send a trade request. To make sure that the trade is received in a reasonable amount of time (less than five minutes, perhaps), the requestor sets the request’s Message Expiration (176) to five minutes. If the messaging system cannot deliver the request in that amount of time, or if the trading application does not receive the message (e.g., read it off of the channel) in time, then the messaging system will take the message off of the trade request channel and put the message on the Dead Letter Channel. The trading system may wish to monitor the system’s Dead Letter Channels to determine if it is missing trades.

---

Guaranteed Delivery

An enterprise is using Messaging (53) to integrate applications.

How can the sender make sure that a message will be delivered even if the messaging system fails?

One of the main advantages of asynchronous messaging over RPC is that the sender, the receiver, and network connecting the two don’t all have to be working at the same time. If the network is not available, the messaging system stores the message until the network becomes available. Likewise, if the receiver is unavailable, the messaging system stores the message and retries delivery until the receiver becomes available. This is the store-and-forward process that messaging is based on. So, where should the message be stored before it is forwarded?

By default, the messaging system stores the message in memory until it can successfully forward the message to the next storage point. This works as long as the messaging system is running reliably, but if the messaging system crashes (for example, because one of its computers loses power or the messaging process aborts unexpectedly), all of the messages stored in memory are lost.

Most applications have to deal with similar problems. All data that is stored in memory is lost if the application crashes. To prevent this, applications use files and databases to persist data to disk so that it survives system crashes. Messaging systems need a similar way to persist messages more permanently so that no message gets lost even if the system crashes.

Use Guaranteed Delivery to make messages persistent so that they are not lost even if the messaging system crashes.

With Guaranteed Delivery, the messaging system uses a built-in datastore to persist messages. Each computer on which the messaging system is installed has its own datastore so that the messages can be stored locally. When the sender sends a message, the send operation does not complete successfully until the message is safely stored in the sender’s datastore. Subsequently, the message is not deleted from one datastore until it is successfully forwarded to and stored in the next datastore. In this way, once the sender successfully sends the message, it is always stored on disk on at least one computer until it is successfully delivered to and acknowledged by the receiver.

Persistence increases reliability but at the expense of performance. Thus, if it’s okay to lose messages when the messaging system crashes or is shut down, avoid using Guaranteed Delivery so messages will move through the messaging system faster.

Also consider that Guaranteed Delivery can consume a large amount of disk space in high-traffic scenarios. If a producer generates hundreds or thousands of messages per second, then a network outage that lasts multiple hours could use up a huge amount of disk space. Because the network is unavailable, the messages have to be stored on the producing computer’s local disk drive, which may not be designed to hold this much data. For these reasons, some messaging systems allow you to configure a retry timeout parameter that specifies how long messages are buffered inside the messaging system. In some high-traffic applications (e.g., streaming stock quotes to terminals), this timeout may have to be set to a short time span, for example, a few minutes. Luckily, in many of these applications, messages are used as Event Messages (151) and can safely be discarded after a short amount of time elapses (see Message Expiration [176]).

It can also be useful to turn off Guaranteed Delivery during testing and debugging. This makes it easy to purge all message channels by stopping and restarting the messaging server. Messages that are still queued up can make it very tedious to debug even simple messaging programs. For example, you may have a sender and a receiver connected by a Point-to-Point Channel (103). If a message is still stored on the channel, the receiver will process that message before any new message that the sender produces. This is a common debugging pitfall in asynchronous, guaranteed messaging. Many commercial messaging implementations also allow you to purge queues individually to allow a fresh restart during testing (see Channel Purger [572]).

With .NET’s MSMQ implementation, for a channel to be persistent, it must be declared transactional, which means senders usually have to be Transactional Clients (484). In JMS, with Publish-Subscribe Channel (106), Guaranteed Delivery only ensures that the messages will be delivered to the active subscribers. To ensure that a subscriber receives messages even when it’s inactive, the subscriber will need to be a Durable Subscriber (522).

---

Example: Stock Trading

In a stock trading system, trade requests and trade confirmations should probably be sent with Guaranteed Delivery to help ensure that none are lost. Change-of-address announcements should be sent with Guaranteed Delivery, but it is probably not necessary with price updates because losing some of them is not significant, and their frequency makes the overhead of Guaranteed Delivery prohibitive.

In Durable Subscriber (522), the stock trading example says that some price-change subscribers may wish to be durable. If so, then perhaps the price-change channel should guarantee delivery as well. Yet other subscribers may not need to be durable or want to suffer the overhead of Guaranteed Delivery. How can these different needs be met? The system may wish to implement two price-change channels, one with Guaranteed Delivery and another without. Only subscribers that require all updates should subscribe to the persistent channel, and their subscriptions should be durable. The publisher may wish to publish updates less frequently on the persistent channel because of its increased overhead. (See the Quality-of-Service Channel strategy discussed under Datatype Channel [111].)

---

---

Example: JMS Persistent Messages

In JMS, message persistence can be set on a per-message basis. In other words, some messages on a particular channel may be persistent, whereas others might not be [JMS 1.1], [Hapner].

When a JMS sender wants to make a message persistent, it uses its MessageProducer to set the message’s JMSDeliveryMode to PERSISTENT. The sender can set persistence on a per-message basis like this:

Session session = // obtain the session 
Destination destination = // obtain the destination 
Message message = // create the message 
MessageProducer producer = session.createProducer(destination); 
producer.send(
     message,
	   javax.jms.DeliveryMode.PERSISTENT,
	   javax.jms.Message.DEFAULT_PRIORITY,
	   javax.jms.Message.DEFAULT_TIME_TO_LIVE);

If the application wants to make all of the messages persistent, it can set that as the default for the message producer.

producer.setDeliveryMode(javax.jms.DeliveryMode.PERSISTENT);

(And, in fact, the default delivery mode for a message producer is persistent.) Now, messages sent by this producer are automatically persistent, so they can simply be sent.

producer.send(message);

Meanwhile, messages sent by other message producers on the same channel may be persistent, depending on how those producers configure their messages.

---

---

Example: IBM WebSphere MQ

In WebSphere MQ, Guaranteed Delivery can be set on a per-channel basis or a per-message basis. If the channel is not persistent, the messages cannot be persistent. If the channel is persistent, the channel can be configured such that all messages sent on that channel are automatically persistent or that an individual message can be sent persistently or nonpersistently.

A channel is configured to be persistent (or not) when it is created in the messaging system. For example, the channel can be configured so that all of its messages will be persistent.

DEFINE Q(myQueue) PER(PERS)

Or the channel can be configured so that the message sender can specify with each message whether the message is persistent or transient.

DEFINE Q(myQueue) PER(APP)

If the channel is set to allow the sender to specify persistency, then a JMS MessageProducer can set that delivery-mode property as described earlier. If the channel is set to make all messages persistent, then the delivery-mode settings specified by the MessageProducer are ignored [WSMQ].

---

---

Example: .NET Persistent Messages

With .NET, persistent messages are created by making a MessageQueue transactional.

MessageQueue.Create("MyQueue", true);

All messages sent on this queue will automatically be persistent [Dickman].

---

Channel Adapter

Many enterprises use Messaging (53) to integrate multiple, disparate applications.

How can you connect an application to the messaging system so that it can send and receive messages?

Most applications were not designed to work with a messaging infrastructure. There are a variety of reasons for this limitation. Many applications were developed as self-contained, standalone solutions even though they contain data or functionality that can be leveraged by other systems. For example, many mainframe applications were designed as a one-in-all application that would never have to interface with other applications. Alas, legacy integration is nowadays one of the most common integration points for enterprise integration solutions. Another reason results from the fact that many message-oriented middleware systems expose proprietary APIs so that an application developer would have to code multiple interfaces to the messaging system, one for each potential middleware vendor.

If applications need to exchange data with other applications, they often are designed to use more generic interface mechanisms such as file exchange or database tables. Reading and writing files is a basic operating system function and does not depend on vendor-specific APIs. Likewise, most business applications already persist data into a database, so little extra effort is required to store data destined for other systems in a database table. Or an application can expose internal functions in a generic API that can be used by any other integration strategy, including messaging.

Other applications may be capable of communicating via a simple protocol like HTTP or TCP/IP. However, these protocols do not provide the same reliability as a Message Channel (60), and the data format used by the application is usually specific to the application and not compatible with a common messaging solution.

In the case of custom applications, we could add code to the application to allow it to send and receive messages. However, this can introduce additional complexity into the application and we need to be careful not to introduce any undesired side effects when making these changes. Also, this approach requires developers to be skilled with both the application logic and the messaging API. Both those approaches also assume that we have access to the application source code. If we deal with a packaged application that we purchased from a third-party software vendor, we may not even have the option of changing the application code.

Use a Channel Adapter that can access the application’s API or data to publish messages on a channel based on this data and that likewise can receive messages and invoke functionality inside the application.

The adapter acts as a messaging client to the messaging system and invokes application functions via an application-supplied interface. Likewise, the Channel Adapter can listen to application-internal events and invoke the messaging system in response to these events. This way, any application can connect to the messaging system and be integrated with other applications as long as it has a proper Channel Adapter.

The Channel Adapter can connect to different layers of the application’s architecture, depending on that architecture and the type of data the messaging system needs to access.

A Channel Adapter Connecting to Different Layers of an Application

1. User Interface Adapter. Sometimes disparagingly called “screen scraping,” these types of adapters can be very effective in many situations. For example, an application may be implemented on a platform that is not supported by the messaging system. Or the owner of the application may have little interest in supporting the integration. This eliminates the option of running the Channel Adapter on the application platform. However, the user interface is usually available from other machines and platforms (e.g., 3270 terminals). Also, the surge of Web-based thin-client architectures has caused a certain revival of user interface integration. HTML-based user interfaces make it very easy to make an HTTP request and parse out the results. Another advantage of user interface integration is that no direct access to the application internals is needed. In some cases, it may not be desirable or possible to expose internal functions of a system to the integration solution. Using a user interface adapter, other applications have the exact same access to the application as a regular user. The downside of user interface adapters is the potential brittleness and low speed of the solution. The application has to parse “user” input and render a screen in response just so that the Channel Adapter can parse the screen back into raw data. This process involves many unnecessary steps, and it can be slow. Also, user interfaces tend to change more frequently than the core application logic. Every time the user interface changes, the Channel Adapter is likely to have to be changed as well.
2. Business Logic Adapter. Most business applications expose their core functions as an API. This interface may be a set of components (e.g., EJBs, COM objects, CORBA components) or a direct programming API (e.g., a C++, C#, or Java library). Since the software vendor (or developer) exposes these APIs expressly for access by other applications, they tend to be more stable than the user interface. In most cases, accessing the API is also more efficient. In general, if the application exposes a well-defined API, this type of Channel Adapter is likely to be the best approach.
3. Database Adapter. Most business applications persist their data inside a relational database. Since the information is already in the database, the Channel Adapter can extract information directly from the database without the application ever noticing, which is a very nonintrusive way to integrate the application. The Channel Adapter can even add a trigger to the relevant tables and send messages every time the data in these tables changes. This type of Channel Adapter can be very efficient and is quite universal, aided by the fact that only two or three database vendors dominate the market for relational databases. This allows us to connect to many different applications with a relatively generic adapter. The downside of a database adapter is that we are poking around deep in the internals of an application. This may not be as risky if we simply read data, but making updates directly to the database can be very dangerous. Also, many application vendors consider the database schema “unpublished,” meaning that they reserve the right to change it at will, which can make a database adapter solution brittle.

An important limitation of Channel Adapters is that they can convert messages into application functions, but they require message formatting that closely resembles the implementation of the components being adapted. For example, a database adapter typically requires the message field names of incoming messages to be the same as the names of tables and fields in the application database. This kind of message format is driven entirely by the internal structure of the application and is not a good message format to use when integrating with other applications. Therefore, most Channel Adapters must be combined with a Message Translator (85) to convert the application-specific message into a message format that complies with the Canonical Data Model (355).

Channel Adapters can often run on a different computer than the application or the database itself. The Channel Adapter can connect to the application logic or the database via protocols such as HTTP or ODBC. While this setup allows us to avoid installing additional software on the application or database server, these protocols typically do not provide the same quality of service that a messaging channel provides, such as Guaranteed Delivery. Therefore, we must be aware that the remote connection to the database can represent a potential point of failure.

Some Channel Adapters are unidirectional. For example, if a Channel Adapter connects to an application via HTTP, it may only be able to consume messages and invoke functions on the application, but it may not be able to detect changes in the application data except through repeated polling, which can be very inefficient.

An interesting variation of the Channel Adapter is the Metadata Adapter, sometimes called Design-Time Adapter. This type of adapter does not invoke application functions but extracts metadata, data that describes the internal data formats of the application. This metadata can then be used to configure Message Translators (85) or to detect changes in the application data formats (see the introduction in Chapter 8, “Message Transformation”). Many application interfaces support this type of metadata extraction. For example, most commercial databases provide a set of system tables that contain metadata in the form of descriptions of the application tables. Likewise, most component frameworks (e.g., J2EE, .NET) provide special “reflection” functions that allow a component to enumerate methods provided by another component.

A special form of the Channel Adapter is the Messaging Bridge (133). The Messaging Bridge (133) connects the messaging system to another messaging system as opposed to a specific application. Typically, a Channel Adapter is implemented as a Transactional Client (484) to ensure that each piece of work the adapter does succeeds in both the messaging system and the other system being adapted.

---

Example: Stock Trading

A stock trading system may wish to keep a log of all of a stocks’ prices in a database table. The messaging system may include a relational database adapter that logs each message from a channel to a specified table and schema. This channel-to-RDBMS adapter is a Channel Adapter. The system may also be able to receive external quote requests from the Internet (TCP/IP or HTTP) and send them on its internal quote-request channel with the internal quote requests. This Internet-to-Channel Adapter is a Channel Adapter.

---

---

Example: Commercial EAI Tools

Commercial EAI vendors provide a collection of Channel Adapters as part of their offerings. Having adapters to all major application packages available greatly simplifies development of an integration solution. Most vendors also provide more generic database adapters as well as software development kits (SDKs) to develop custom adapters.

---

---

Example: Legacy Platform Adapters

A number of vendors provide adapters from common messaging system to legacy systems executing on platforms such as UNIX, MVS, OS/2, AS/400, Unisys, and VMS. Most of these adapters are specific to a certain messaging system. For example, Envoy Technologies’ EnvoyMQ is a Channel Adapter that connects many legacy platforms with MSMQ. It consists of a client component that runs on the legacy computer and a server component that runs on a Windows computer with MSMQ.

---

---

Example: Web Services Adapters

Many messaging systems provide Channel Adapters to move SOAP messages between an HTTP transport and the messaging system. This way, SOAP messages can be transmitted over an intranet using a reliable, asynchronous messaging system and over the global Internet (and through firewalls) using HTTP. One example of such an adapter is the Web Services Gateway for IBM’s WebSphere Application Server.

---

Messaging Bridge

An enterprise is using Messaging (53) to enable applications to communicate. However, the enterprise uses more than one messaging system, which confuses the issue of which messaging system an application should connect to.

How can multiple messaging systems be connected so that messages available on one are also available on the others?

A common problem is an enterprise that uses more than one messaging system. This can occur because of a merger or acquisition between two different companies that have standardized around different messaging products. Sometimes a single enterprise that uses one messaging system to integrate its mainframe/legacy systems chooses another for its J2EE or .NET web application servers and then needs to integrate the two messaging systems. Another common occurrence is an application that participates as part of multiple enterprises, such as a B2B client that wants to be a bidder in multiple auctioning systems. If the various auction clusters use different messaging systems, the bidder applications within an enterprise may wish to consolidate the messages from several external messaging systems onto a single internal messaging system. Another example is the extremely large enterprise with a huge number of Message Channels (60) and Message Endpoints (95) that may require more than one instance of the messaging system, which means those instances must be connected somehow.

If the messages on one system are of no interest to the applications using the other messaging system, then the systems can remain completely separate. But because the applications are part of the same enterprise, often some applications using one messaging system will be interested in messages being transmitted on another messaging system.

A common misconception is that a standardized messaging API such as JMS solves this problem; it does not. JMS makes two compliant messaging systems look the same to a client application, but it does nothing to make the two messaging systems work with each other. For the messaging systems to work together, they need to be interoperable, meaning that they use the same message format and transmit a message from one message store to the next in the same way. Messaging systems from two different vendors are rarely interoperable; a message store from one vendor can work only with other message stores from the same vendor.

Each application in the enterprise could choose to implement a client for each messaging system in the enterprise, but that would increase complexity and duplication in the messaging layer. This redundancy would become especially apparent if the enterprise added yet another messaging system and all of the applications had to be modified. On the other hand, each application could choose to interface with only one messaging system and ignore data on the other messaging systems. This would make the application simpler but could cause it to ignore a great deal of enterprise data. What is needed is a way for messages on one messaging system that are of interest to applications on another messaging system to be made available on the second messaging system as well.

Use a Messaging Bridge, a connection between messaging systems that replicates messages between systems.

Typically, there is no practical way to connect two complete messaging systems, so instead we connect individual, corresponding channels between the messaging systems. The Messaging Bridge is a set of Channel Adapters (127), where the non-messaging client is actually another messaging system and where each pair of adapters connects a pair of corresponding channels. The bridge acts as a map from one set of channels to the other and transforms the message format of one system to the other. The connected channels may be used to transmit messages between traditional clients of the messaging system or strictly for messages intended for other messaging systems.

You may need to implement the Messaging Bridge yourself for your enterprise. The bridge is a specialized Message Endpoint (95) application that is a client of both messaging systems. When a message is delivered on a channel of interest in one messaging system, the bridge consumes the message and sends another with the same contents on the corresponding channel in the other messaging system.

Many messaging system vendors have product extensions for bridging to messaging systems from other vendors. Thus, you may be able to buy a solution rather than build it yourself.

If the other “messaging system” is really a simpler protocol, such as HTTP, apply the Channel Adapter (127) pattern.

Messaging Bridge is necessary because different messaging system implementations have their own proprietary approaches for how to represent messages and how to forward them from one store to the next. Web services may be standardizing this, such that two messaging system installs, even from different vendors, may be able to act as one by transferring messaging using web services standards. See the discussion of WS-Reliability and WS-ReliableMessaging in Chapter 14, “Concluding Remarks.”

---

Example: Stock Trading

A brokerage house may have one messaging system that the applications in its various offices use to communicate. A bank may have a different messaging system that the applications in its various branches use to communicate. If the brokerage and the bank decide to merge into a single company that offers bank accounts and investment services, which messaging system should the combined company use? Rather than redesigning half of the company’s applications to use the new messaging system, the company can use a Messaging Bridge to connect the two messaging systems. This way, for example, a banking application and a brokerage application can coordinate to transfer money between a savings account and a securities trading account.

---

---

Example: MSMQ Bridges

MSMQ defines an architecture based on connector servers that enables connector applications to send and receive messages using other (non-MSMQ) messaging systems. An MSMQ application using a connector server can perform the same operations on channels from other messaging systems that it can perform on MSMQ channels [Dickman].

Microsoft’s Host Integration Server product contains an MSMQ-MQSeries Bridge service that makes the two messaging systems work together. It lets MSMQ applications send messages via MQSeries channels and vice versa, making the two messaging systems act as one.

Envoy Technologies, licenser of the MSMQ-MQSeries Bridge, also has a related product called Envoy Connect. It connects MSMQ and BizTalk servers with messaging servers running on non-Windows platforms, especially the J2EE platform, coordinating J2EE and .NET messaging within an enterprise.

---

---

Example: SonicMQ Bridges

Sonic Software’s SonicMQ has SonicMQ Bridge products that support IBM MQSeries, TIBCO TIB/Rendezvous, and JMS. This enables messages on Sonic channels to be transmitted on other messaging systems’ channels as well.

---

Message Bus

An enterprise contains several existing systems that must be able to share data and operate in a unified manner in response to a set of common business requests.

What architecture enables separate applications to work together but in a decoupled fashion such that applications can be easily added or removed without affecting the others?

An enterprise often contains a variety of applications that operate independently but must work together in a unified manner. Enterprise Application Integration (EAI) defines a solution to this problem but doesn’t describe how to accomplish it.

For example, consider an insurance company that sells different kinds of insurance products (life, health, auto, home, etc.). As a result of corporate mergers and of the varying winds of change in IT development, the enterprise consists of a number of separate applications for managing the company’s various products. An insurance agent trying to sell a customer several different types of policies must log onto a separate system for each policy, wasting effort and increasing the opportunity for mistakes.

The agent needs a single, unified application for selling customers a portfolio of policies. Other types of insurance company employees, such as claims adjusters and customer service representatives, need their own applications for working with the insurance products, but they also want their applications to present a unified view. The individual product applications must be able to work together, perhaps to offer a discount when purchasing more than one policy and to process a claim that is covered by more than one policy.

The IT department could rewrite the product applications to all use the same technology and work together, but the amount of time and money to replace systems that already work (even though they don’t work together) is prohibitive. IT could create a unified application for the agents, but this application needs to connect to the systems that actually manage the policies. Rather than unifying the systems, this new application creates one more system that doesn’t integrate with the others.

The agent application could integrate with all of these other systems, but that would make it much more complex. The complexity would be duplicated in the applications for claims adjusters and customer service representatives. Furthermore, these unified user applications would not help the product applications integrate with each other.

Even if all of these applications could be made to work together, any change to the enterprise’s configuration could make it all stop working. Not all applications will be available all of the time, yet the ones that are running must be able to continue with minimal impact from those that are not running. Over time, applications will need to be added to and removed from the enterprise, with minimal impact on the other applications. What is needed is an integration architecture that enables the product applications to coordinate in a loosely coupled way and for user applications to be able to integrate with them.

Structure the connecting middleware between these applications as a Message Bus that enables them to work together using messaging.

A Message Bus is a combination of a Canonical Data Model (355), a common command set, and a messaging infrastructure to allow different systems to communicate through a shared set of interfaces. This is analogous to a communications bus in a computer system, which serves as the focal point for communication between the CPU, main memory, and peripherals. Just as in the hardware analogy, there are a number of pieces that come together to form the message bus.

1. Common communication infrastructure—Just as the physical pins and wires of a PCI bus provide a common, well-known physical infrastructure for a PC, a common infrastructure must serve the same purpose in a message bus. Typically, a messaging system is chosen to serve as the physical communications infrastructure, providing a crossplatform, cross-language universal adapter between the applications. The infrastructure may include Message Router (78) capabilities to facilitate the correct routing of messages from system to system. Another common option is to use Publish-Subscribe Channels (106) to facilitate sending messages to all receivers.
2. Adapters—The different systems must find a way to interface with the Message Bus. Some applications may be ready-built to connect to the bus, but most will need adapters to connect to the messaging system. These adapters are commonly commercial or custom Channel Adapters (127) and Service Activators (532). They may be specialized to handle tasks like invoking CICS transactions with the proper parameters or converting the bus’s general data structures to the specific representation an application uses. This also requires a Canonical Data Model (355) that all systems can agree on.
3. Common command structure—Just as PC architectures have a common set of commands to represent the different operations possible on the physical bus (read bytes from an address, write bytes to an address), there must be common commands that all the participants in the Message Bus can understand. Command Message (145) illustrates how this feature works. Another common implementation for this is the Datatype Channel (111), where a Message Router (78) makes an explicit decision as to how to route particular messages (like purchase orders) to particular endpoints. It is at the end that the analogy breaks down, since the level of the messages carried on the bus are much more fine-grained than the “read/write” kinds of messages carried on a physical bus.

In our EAI example, a Message Bus could serve as a universal connector between the various insurance systems and as a universal interface for client applications that wish to connect to the insurance systems.

Here we have two GUIs that know only about the Message Bus; they are entirely unaware of the complexities of the underlying systems. The bus is responsible for routing Command Messages (145) to the proper underlying systems. In some cases, the best way to handle the command messages is to build an adapter to the system that interprets the command and then communicates with the system in a way it understands (invoking a CICS transaction, for instance, or calling a C++ API). In other cases, it may be possible to build the command-processing logic directly into the existing system as an additional way to invoke current logic.

Once the Message Bus has been developed for the agent GUI, it is easy to reuse for other GUIs, such as those for claims processors, customer service representatives, and customers who use a web interface for to browse their own accounts. The features and security control of these GUI applications differ, but their need to work with the back-end applications is the same.

A Message Bus forms a simple, useful service-oriented architecture for an enterprise. Each service has at least one request channel that accepts requests of an agreed-upon format and probably a corresponding reply channel that supports a specified reply format. Any participant application can use these services by making requests and waiting for replies. The request channels, in effect, act as a directory of the services available.

A Message Bus requires that all of the applications using the bus use the same Canonical Data Model (355). Applications adding messages to the bus may need to depend on Message Routers (78) to route the messages to the appropriate final destinations. Applications not designed to interface with a messaging system may require Channel Adapters (127) and Service Activators (532).

---

Example: Stock Trading

A stock trading system may wish to offer a unified suite of services including stock trades, bond auctions, price quotes, portfolio management, and so on. This may require several separate back-end systems that have to coordinate with each other. To unify the services for a front-end customer GUI, the system could employ an intermediate application that offered all of these services and delegated their performance to the back-end systems. The back-end systems could even coordinate through this intermediary application. However, the intermediary application would tend to become a bottleneck and a single point of failure.

Rather than an intermediary application, a better approach might be a Message Bus with channels for requesting various services and getting responses. This bus could also enable back-end systems to coordinate with each other. A front-end system could simply connect to the bus and use it to invoke services. The bus could be distributed relatively easily across multiple computers to provide load distribution and fault tolerance.

Once the Message Bus is in place, connecting front-end GUIs would be easy; they each just need to send and receive messages from the proper channels. One GUI might enable a retail broker to manage his customers’ portfolios. Another web-based GUI could enable any customer with a web browser to manage his own portfolio. Another non-GUI front end might support personal finance programs like Intuit’s Quicken and Microsoft’s Money, enabling customers using those programs to download trades and current prices. Once the Message Bus is in place, developing new user applications is much simpler.

Likewise, the trading system may want to take advantage of new back-end applications, such as by switching one trading application for another or spreading price quote requests across multiple applications. Implementing a change like this is as simple as adding and removing applications from the Message Bus. Once the new applications are in place, none of the other applications have to change; they just keep sending messages on the bus’s channels as usual.

---