Messaging planning

Key points to consider when you plan the messaging aspects of your Eclipse Amlen solution.

Protocols

Eclipse Amlen natively provides support for MQTT and JMS, and can convert between the two protocols. You must consider which protocols are appropriate for your business needs. For more information about the supported protocols, see Supported protocols. For more information about converting between the protocols, see Conversion between MQTT and JMS.

MQTT and JMS support different qualities of service. In general, higher qualities of service are slower than lower qualities of service. For more information about the way MQTT and JMS handle the delivery of messages, see Message delivery.

Message size

If you send large messages with a high quality of service level, the Eclipse Amlen server cannot store many messages before the available storage is filled. Consider the volume and size of the messages you want to send, and whether the messages must be persistent. These decisions affect the number of messages that Eclipse Amlen can handle.

You can limit the message size that is allowed to be sent through the Eclipse Amlen by using the maximum message size attribute of an endpoint. For more information about endpoints, see Message hubs, endpoints, and policies.

Message count

For publish/subscribe messaging, you can configure how many messages the Eclipse Amlen server stores, by setting the Max Messages attribute of a messaging policy. The server stores the messages until the messages are received by subscribers at the reliability level required. If a subscriber that requires an exactly once level of reliability cannot be reached, the message is stored.

You can choose what happens when the maximum message count is reached by using the Max Messages Behavior attribute of a messaging policy. You can either reject new messages, or you can discard a percentage of old messages that are stored on the server. If you choose to reject new messages, and the maximum message count is reached, messages cannot be sent. If you choose to discard old messages, and the maximum message count is reached, old messages are removed from the server. Any messages that are being delivered, that is, messages that are sent but not yet acknowledged, are not removed. If a retained message is one of the old messages, it is removed. The quality of service, persistence, and priority of the messages does not prevent the messages from being removed. Therefore, if all messages must be received in order for your applications to function correctly, you cannot set the max messages behavior to discard old messages. You must consider which behavior to use, based on your messaging requirements.

For more information about messaging policies, see Authorizing client messaging.

Max Messages Behavior

You can choose what happens when the maximum message count is reached by using the Max Messages Behavior attribute of a messaging policy. You can either reject new messages, or you can discard a percentage of old messages that are stored on the server. If you choose to reject new messages, and the maximum message count is reached, messages cannot be sent. If you choose to discard old messages, and the maximum message count is reached, old messages are removed from the server. Any messages that are being delivered, that is, messages that are sent but not yet acknowledged, are not removed. If a retained message is one of the old messages, it is removed. The quality of service, persistence, and priority of the messages does not prevent the messages from being removed. Therefore, if all messages must be received in order for your applications to function correctly, you cannot set the max messages behavior to discard old messages. You must consider which behavior to use, based on your messaging requirements.

The value of Max Messages Behavior is determined by the messaging policy that is associated with the subscription. To determine the messaging policy that is associated with the subscription, you can either use the Monitoring page in the Amlen WebUI, or the REST API GET method with the following Eclipse Amlen URI:
http://<admin-endpoint-IP:Port>/ima/v1/monitor/Subscription
  • For non-durable shared subscriptions, Max Messages Behavior is determined by the value set for the subscriber that created the shared subscription. The setting applies to all subsequent subscribers, regardless of their own setting of Max Messages Behavior. If the subscription is deleted, perhaps because all subscribers disconnected, and then is re-created by a different subscriber, the subscription might have a different setting for Max Messages Behavior.
  • For durable shared subscriptions, Max Messages Behavior is determined by the value set for the subscriber that created the durable shared subscription.
  • For global-shared durable subscriptions, Max Messages Behavior is determined by the value set on the messaging policy with a type of Subscription for the subscriber that created the global-shared durable subscription.
  • For all types of subscriptions, you can update Max Messages Behavior on the associated messaging policy; all subscriptions that are associated with that messaging policy are then updated to use the new value.

The use of wildcard characters is significant. A number of different subscriptions that are using different messaging policies with different values of Max Messages Behavior can subscribe to the same topic at the same time; they are part of the same topic community. It is the value of the Max Messages Behavior attribute of the subscription the buffer of which is full at the time of a publish that dictates the behavior for the entire topic community. A subscription that is using a messaging policy with a value of Reject New Messages in the Max Messages Behavior attribute causes publish requests to fail when its buffer becomes full. Even if other subscriptions in the topic community are using messaging policies with a value of Discard Old Messages in the Max Messages Behavior attribute, their publish requests fail.

Consider the following example scenarios:

  1. There are a number of subscriptions to the topic /RiverGauges/*; note the use of the wildcard. The buffer of one of these subscriptions becomes full. It is using a messaging policy with a Max Messages Behavior attribute set to Reject New Messages. All publish requests to any topic in the /RiverGauges/* topic community fail. Even if other subscriptions are using messaging policies with a Max Messages Behavior attribute set to Discard Old Messages, publish requests fail. Publish requests continue to fail until the subscriber with the full buffer starts to receive messages and space is created in its buffer.

    If, however, the setting of the Max Messages Behavior attribute of the subscription to the topic /RiverGauges/* is set to Discard Old Messages, and its buffer becomes full, other subscriptions are not affected. Messages are deleted from the buffer of the subscriber with the full buffer. Gaps in the incoming message stream might be experienced as a result of messages being deleted from its buffer.

  2. There are a number of subscriptions to the /RiverGauges/Mississippi topic. The buffer of one of these subscriptions becomes full. It is using a messaging policy with a Max Messages Behavior attribute set to Reject New Messages. All publish requests to the /RiverGauges/Mississippi topic fail. Even if subscriptions are using messaging policies with a Max Messages Behavior attribute set to Discard Old Messages, publish requests fail. All subscriptions to the /RiverGauges/Mississippi topic cannot receive any new messages. Subscriptions to the /RiverGauges/* topic community continue to receive messages for all topics except for messages for the /RiverGauges/Mississippi topic. Publish requests continue to fail until the subscriber with the full buffer starts to receive messages and space is created in its buffer.

    If, however, the setting of the Max Messages Behavior attribute of the subscriber to the topic /RiverGauges/Mississippi is set to Discard Old Messages, and its buffer becomes full, other subscriptions are not affected. Messages are deleted from the buffer of the subscriber with the full buffer. Gaps in the incoming message stream might be experienced as a result of messages being deleted from its buffer.

If you choose to discard old messages, the quality of service, persistence, priority, expiry status, and reliability of the messages are not considered and do not prevent the messages from being removed.

To be published successfully, MQTT messages with QoS of 1 or 2, or JMS messages with the equivalent qualities of service, must be sent to all subscriptions in the topic community. If a subscription has a full buffer and a Max Messages Behavior attribute set to Discard Old Messages, the message is always accepted and then, if necessary, messages in the buffer are discarded. If the Max Messages Behavior attribute is set to Reject New Messages, the publish request fails for all subscriptions in the topic community.

If MQTT messages with QoS of 0, or JMS messages with the equivalent quality of service, are sent to a subscription that has a full buffer and a Max Messages Behavior attribute set to Discard Old Messages, the publish request succeeds but some existing messages are discarded from the buffer. If the Max Messages Behavior attribute is set to Reject New Messages, the publish request succeeds but other incoming messages are rejected.

Message order

If the message arrival order is important to your solution, you must configure your solution to ensure that message order is preserved. To preserve the message order, your solution must meet the following requirements:
  • The producer or publisher must be single threaded.
  • Messages must be exchanged with the same quality of service.
  • If you are using MQ Connectivity, you must map to a single queue manager.
  • For publish/subscribe systems, the messages must be published to a single topic.

Topic Namespace

Maximum number of topic levels
Eclipse Amlen limits topic strings to 32 levels. Therefore, you must consider your publish/subscribe topology to ensure that the number of topic levels does not exceed 32.
Topic string name restrictions
${UserID}, ${GroupID}, ${ClientID}, and ${CommonName} are reserved for variable substitution in messaging policies and cannot be used in a topic string.

Retained messages

You can use retained messages to send the most recent message to subscribers that subscribe after that message was published.

Only one message is retained for each topic, so existing retained messages are deleted when a new retained message is published to the topic. The deletion of the existing message might not occur synchronously with the arrival of the new retained message. In the case where more than one publisher publishes retained messages to a single topic, it is not possible to know which message is stored. It is possible that an earlier message sent by one publisher is stored as the retained message, rather than a later message sent by another publisher. Therefore, where possible, have no more than one publisher sending retained messages to a topic.

Messages are not retained after Eclipse Amlen is restarted if the messages are not sent as persistent retained messages. Consider sending retained messages as persistent messages so that the messages continue to be available to new subscribers even if Eclipse Amlen is restarted.

Messaging users and groups

When you create messaging policies and connection policies, you can restrict access based on messaging user ID and messaging group membership. You can use an LDAP server connection to provide the details of the users and groups. You can connect only one LDAP server to Eclipse Amlen. Therefore, you must consider how your existing infrastructure might be used for messaging users and groups.

When you use the ${GroupID} variable in your messaging policies, ensure that, where the same user ID is being used in more than one group with the same group ID and the user ID is referring to different users, you consider having unique group IDs. Unique group IDs help to ensure that access is restricted to only those users that are intended to have access. For more information about using variable substitution, see Variable substitution in messaging policies.

Expired messages

If it is important that subscribing client applications do not process expired messages, it is good practice for the subscribing client application to check the expiry time of messages that they receive from the client. The subscribing client application can then discard those messages that have expired. If the subscribing client application needs to perform this check of expiry time, it is important that the subscriber host clock is synchronized with the Eclipse Amlen server host clock. Synchronization of the clocks is required for the expiration check to work correctly. Use a network time protocol (NTP) server to keep the clocks synchronized.

LTPA or OAuth authentication

For clients that depend on LTPA or OAuth authentication, it is important that Eclipse Amlen server time is synchronized with the LTPA token provider or OAuth token provider. The authentication request from the Eclipse Amlen client might fail because of the expiry of the LTPA token or OAuth token if the clocks are not synchronized.