jms-spec
  1. jms-spec
  2. JMS_SPEC-40

Allow multiple consumers to be created on the same topic subscription

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.1
    • Fix Version/s: 2.0PD, 2.0
    • Labels:
      None

      Description

      This proposal would allow multiple message consumers to be used to consume messages from the same durable or non-durable topic subscription. This would increase scalability by allowing messages from a topic to be processed by multiple consumers in a Java SE environment. Note that the existing JMS 1.1 API already allows this to be achieved for messages from a queue.

      The multiple consumers might be in the same or different JVMs. Within the same JVM, the multiple consumers might use the same or different sessions and connection. Note, however, that the normal restrictions on the use of a session by multiple threads would continue to apply.

      Durable subscriptions

      For durable subscriptions, there would be no need to change the existing API. Applications would simply be allowed to create multiple consumers on the same durable subscription:

      This affects the following JMS existing JMS 2.1 methods on Session (and its subtypes QueueSession, TopicSession, XASession, XAQueueSession and XATopicSession):

      createDurableSubscriber(Topic topic, String name)
      createDurableSubscriber(Topic topic, String name, String messageSelector, boolean noLocal) 
      

      It also affects the following JMS new JMS 2.0 methods on Session (and its subtypes QueueSession, TopicSession, XASession, XAQueueSession and XATopicSession) (added in JIRA issue 51 ):

      createDurableConsumer(Topic topic, String name)
      createDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal)
      

      And in the new JMS 2.0 simplified API, it also affects the following method on JMSContext:

      createDurableConsumer(Topic topic, String name)
      createDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal)
      

      All these methods currently have a restriction "Only one session at a time can have a TopicSubscriber for a particular durable subscription.". This restriction would be removed.

      No change is proposed to the existing JMS requirement that only one connection may have a given clientID. This is defined in JMS 1.1 Section 4.4.3.2 "Client identifier", which states:

      By definition, the client state identified by a client identifier can be 'in use' by only one client at a time. A JMS provider must prevent concurrently executing clients from using it.

      This means that if a durable subscription is created by a connection with clientId set then it will not be possible to create a second consumer on it using a different connection.

      However in JMS 2.0, clientId will become optional when creating a durable subscription (see JIRA issue 39). If a durable subscription is created by a connection with clientId unset then other connections may freely create further consumers on the same durable subscription.

      This means that for two message consumers on different connections to share a durable subscription, clientId would need to be left unset.

      In addition, the effect of using a different topic or message selector than was used when the subscription was first created needs to be amended. Currently this states (from the javadoc for Session.createDurableSubscriber):

      A client can change an existing durable subscription by creating a durable TopicSubscriber with the same name and a new topic and/or message selector. Changing a durable subscriber is equivalent to unsubscribing (deleting) the old one and creating a new one.

      This behaviour will continue, but only if there are no other active consumers on the durable subscription. If there is an active consumer on the durable subscription, and an attempt is made to create an additional consumer with a different topic or message selector, then a JMSException will be thrown (in the case of JMSContext a JMSRuntimeException will be thrown).

      The semantics for the unsubscribe methods on Session and JMSContext would remain unchanged. The javadocs state that "It is erroneous for a client to delete a durable subscription while there is an active MessageConsumer or TopicSubscriber for the subscription". This will continue to be the case.

      Non-durable subscriptions

      For non-durable subscriptions, the JMS 1.1 specification specifies that if two non-durable TopicSubscribers are created on the same topic then two independent subscriptions are created: a copy of each message is sent to every consumer on that topic (with an appropriate message selector).

      To allow two consumers to share the same subscription, so that they can share the load of processing messages from that subscription, then new API is required. To allow the consumers to specify that they wish to use the same non-durable subscription, they will need to specify the same sharedSubscriptionName.

      The following new methods on Session (and its subtypes) are proposed:

      MessageConsumer createSharedConsumer(
         Topic topic, String sharedSubscriptionName);
      MessageConsumer createSharedConsumer(
         Destination destination, String sharedSubscriptionName, String messageSelector)
      MessageConsumer createSharedConsumer(
         Destination destination, String sharedSubscriptionName, String messageSelector, boolean NoLocal)
      

      Note that if these methods were called simply createConsumer there would be a clash with the existing method createConsumer(Destination destination, String messageSelector). Note that these methods take a Topic, not a Destination.

      In the new simplified API, the following new methods on JMSContext are proposed:

      JMSConsumer createSharedConsumer(
         Topic topic, String sharedSubscriptionName)
      JMSConsumer createSharedConsumer(
         Topic topic, String sharedSubscriptionName, String messageSelector)
      JMSConsumer createSharedConsumer(
         Topic topic, String sharedSubscriptionName, String messageSelector, boolean noLocal)
      

      Note that a given shared subscription does not exist if there are no consumers using it.

        Issue Links

          Activity

          Hide
          Nigel Deakin added a comment -

          The JMS 2.0 expert group has agreed that the noLocal parameter serves no useful purpose for shared subscriptions (both durable and non-durable) and should be removed.

          Changes to Session:

          Removed:

          • createSharedConsumer(Topic topic, String sharedSubscriptionName, java.lang.String messageSelector, boolean noLocal)

          Changed:

          • createSharedDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal)

          To:

          • createSharedDurableConsumer(Topic topic, String name, String messageSelector)

          Changes to JMSContext

          Removed:

          • createSharedConsumer(Topic topic, String sharedSubscriptionName, java.lang.String messageSelector, boolean noLocal)

          Changed:

          • createSharedDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal)

          To:

          • createSharedDurableConsumer(Topic topic, String name, String messageSelector)
          Show
          Nigel Deakin added a comment - The JMS 2.0 expert group has agreed that the noLocal parameter serves no useful purpose for shared subscriptions (both durable and non-durable) and should be removed. Changes to Session : Removed: createSharedConsumer(Topic topic, String sharedSubscriptionName, java.lang.String messageSelector, boolean noLocal) Changed: createSharedDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal) To: createSharedDurableConsumer(Topic topic, String name, String messageSelector) Changes to JMSContext Removed: createSharedConsumer(Topic topic, String sharedSubscriptionName, java.lang.String messageSelector, boolean noLocal) Changed: createSharedDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal) To: createSharedDurableConsumer(Topic topic, String name, String messageSelector)
          Hide
          Nigel Deakin added a comment - - edited

          I have updated the API for creating durable subscriptions to retain backwards compatibility with JMS 1.1 as follows:

          The following methods have been reverted to the JMS 1.1 behaviour:

          Methods on Session:

          • TopicSubscriber createDurableSubscriber(Topic topic, String name) (JMS 1.1)
          • TopicSubscriber createDurableSubscriber(Topic topic, String name, String messageSelector, boolean noLocal) (JMS 1.1)
          • MessageConsumer createDurableConsumer(Topic topic, String name) (JMS 2.0)
          • MessageConsumer createDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal) (JMS 2.0)

          Methods on JMSContext:

          • JMSConsumer createDurableConsumer(Topic topic, String name)
          • JMSConsumer createDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal)

          These methods:

          • Will only allow a single consumer on a durable subscription
          • clientId MUST be set

          I have also added the following new methods to support shared durable subscriptions:

          Methods on Session:

          • MessageConsumer createSharedDurableConsumer(Topic topic, String name)
          • MessageConsumer createSharedDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal)

          Methods on JMSContext:

          • JMSConsumer createSharedDurableConsumer(Topic topic, String name)
          • JMSConsumer createSharedDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal)

          These methods:

          • Will allow multiple consumers on a durable subscription
          • clientId may or may not be set

          You can see the updated API docs here:
          http://jms-spec.java.net/2.0-SNAPSHOT/apidocs/index.html

          I have also updated the spec to incorporate these changes:
          http://java.net/projects/jms-spec/sources/repository/content/jms2.0/specification/word/JMS20.pdf

          Show
          Nigel Deakin added a comment - - edited I have updated the API for creating durable subscriptions to retain backwards compatibility with JMS 1.1 as follows: The following methods have been reverted to the JMS 1.1 behaviour: Methods on Session : TopicSubscriber createDurableSubscriber(Topic topic, String name) (JMS 1.1) TopicSubscriber createDurableSubscriber(Topic topic, String name, String messageSelector, boolean noLocal) (JMS 1.1) MessageConsumer createDurableConsumer(Topic topic, String name ) (JMS 2.0) MessageConsumer createDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal) (JMS 2.0) Methods on JMSContext : JMSConsumer createDurableConsumer(Topic topic, String name) JMSConsumer createDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal) These methods: Will only allow a single consumer on a durable subscription clientId MUST be set I have also added the following new methods to support shared durable subscriptions: Methods on Session : MessageConsumer createSharedDurableConsumer(Topic topic, String name) MessageConsumer createSharedDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal) Methods on JMSContext : JMSConsumer createSharedDurableConsumer(Topic topic, String name) JMSConsumer createSharedDurableConsumer(Topic topic, String name, String messageSelector, boolean noLocal) These methods: Will allow multiple consumers on a durable subscription clientId may or may not be set You can see the updated API docs here: http://jms-spec.java.net/2.0-SNAPSHOT/apidocs/index.html I have also updated the spec to incorporate these changes: http://java.net/projects/jms-spec/sources/repository/content/jms2.0/specification/word/JMS20.pdf
          Hide
          Nigel Deakin added a comment - - edited

          Reopening this issue as this feature changes the behaviour of Session.createDurableSubscription in an incompatible way

          Incompatibility 1.

          According to JMS 1.1 section 6.11.1: "Only one session at a time can have a TopicSubscriber for a particular durable subscription.". The spec and API docs do not explicitly state that an attempt to create two {{TopicSubscriber}}s on the same durable subscription should throw an exception, but I think this is implied, at least for the case where different sessions are used.

          In JMS 2.0 we are proposing to remove this restriction and allow multiple {{TopicSubscriber}}s(and {{MessageConsumer}}s) to be created on the same durable subscription.

          Incompatibility 2.

          According to JMS 1.1 section 6.11.1: "Sessions with durable subscribers must always provide the same client
          identifier." The spec and API docs do not explicitly state that clientId must be set when calling createDurableSubscription, and that an exception must be thrown if it is not set, but again I think this is is implied.

          In JMS 2.0 we are proposing to remove this restriction and allow clientId to be either set or unset when Session.createDurableSubscription is called. If clientId is set then the combination of clientId and subscription name will be used to identify the durable subscription. If clientId is not set then the subscription name will be used to identify the durable subscription.

          Show
          Nigel Deakin added a comment - - edited Reopening this issue as this feature changes the behaviour of Session.createDurableSubscription in an incompatible way Incompatibility 1. According to JMS 1.1 section 6.11.1: "Only one session at a time can have a TopicSubscriber for a particular durable subscription.". The spec and API docs do not explicitly state that an attempt to create two {{TopicSubscriber}}s on the same durable subscription should throw an exception, but I think this is implied, at least for the case where different sessions are used. In JMS 2.0 we are proposing to remove this restriction and allow multiple {{TopicSubscriber}}s(and {{MessageConsumer}}s) to be created on the same durable subscription. Incompatibility 2. According to JMS 1.1 section 6.11.1: "Sessions with durable subscribers must always provide the same client identifier." The spec and API docs do not explicitly state that clientId must be set when calling createDurableSubscription , and that an exception must be thrown if it is not set, but again I think this is is implied. In JMS 2.0 we are proposing to remove this restriction and allow clientId to be either set or unset when Session.createDurableSubscription is called. If clientId is set then the combination of clientId and subscription name will be used to identify the durable subscription. If clientId is not set then the subscription name will be used to identify the durable subscription.
          Hide
          Nigel Deakin added a comment - - edited

          Because shared non-durable subscriptions are so similar in concept to durable subscriptions I think we should aim to make the definition of noLocal for the two types of subscription as similar as possible. This means:

          1. Changing the definition of a shared non-durable subscription so that the client identifier, if set, forms part of the name of the shared non-durable subscription (as is already the case with durable subscriptions).

          2. Defining noLocal in exactly the same way for shared non-durable subscriptions as for durable subscriptions:

          When a durable or shared non-durable subscription is created on a topic, the noLocal argument may be used to specify that messages published to the topic by its own connection or any other with the same client identifier will not be added to the durable subscription. If the client identifier is unset then setting noLocal to true will cause an exception to be thrown.

          Since only one connection may have a given client identifier at a time, this means that the noLocal flag may only be used when there is a single connection consuming from a subscription. I believe this is a benefit as it it gives us a clear, simple definition, it maintains consistency with JMS 1.1, and avoiding the need to contrive complex and entirely hypothetical use cases in which noLocal is used in conjunction with multiple consumers.

          Following consultation with the expert group and user alias I have now added this definition to the draft specification.

          To see the new text in the spec, please download the latest draft here (log in to java.net first) and read section 6.11.2. "Shared non-durable subscriptions"

          You can find the updated API documentation for Session.createSharedConsumer here and for JMSContext.createSharedConsumer here

          I have added a special note to section A.3. "Unresolved issues in the JMS 2.0 Public Draft" to state that this is an issue which we aren't completely sure about, and that when this specification goes to public review we're particularly welcome comments on it.

          Show
          Nigel Deakin added a comment - - edited Because shared non-durable subscriptions are so similar in concept to durable subscriptions I think we should aim to make the definition of noLocal for the two types of subscription as similar as possible. This means: 1. Changing the definition of a shared non-durable subscription so that the client identifier, if set, forms part of the name of the shared non-durable subscription (as is already the case with durable subscriptions). 2. Defining noLocal in exactly the same way for shared non-durable subscriptions as for durable subscriptions: When a durable or shared non-durable subscription is created on a topic, the noLocal argument may be used to specify that messages published to the topic by its own connection or any other with the same client identifier will not be added to the durable subscription. If the client identifier is unset then setting noLocal to true will cause an exception to be thrown. Since only one connection may have a given client identifier at a time, this means that the noLocal flag may only be used when there is a single connection consuming from a subscription. I believe this is a benefit as it it gives us a clear, simple definition, it maintains consistency with JMS 1.1, and avoiding the need to contrive complex and entirely hypothetical use cases in which noLocal is used in conjunction with multiple consumers. Following consultation with the expert group and user alias I have now added this definition to the draft specification. To see the new text in the spec, please download the latest draft here (log in to java.net first) and read section 6.11.2. "Shared non-durable subscriptions" You can find the updated API documentation for Session.createSharedConsumer here and for JMSContext.createSharedConsumer here I have added a special note to section A.3. "Unresolved issues in the JMS 2.0 Public Draft" to state that this is an issue which we aren't completely sure about, and that when this specification goes to public review we're particularly welcome comments on it.
          Hide
          Nigel Deakin added a comment -

          To Answer Chris's comment of 13/Sep/12 07:13 PM:

          1. My option B was that if noLocal was true then once a connection has created a consumer on the subscription its messages would no longer be added to the subscription. This means they would not be delivered to any consumers. I think that was what you were asking for. As you say, if you have multiple consumers on the same subscription they are cooperating to process the messages so they should behave the same way.

          2. You ask what would happen if "a shared consumer is created with noLocal=false, then a second is created on the same connection with noLocal=true (or vice versa). Which one wins?"

          If the first session still had a consumer on the shared non-durable subscription, then if a second connection or session tried to create a consumer on the same subscription with a different topic, message selector or noLocal value then they are trying to change the nature of the subscription, so it should throw an exception.

          Obviously if the first connection was closed before the second connection created the consumer then this would not be a problem, as the subscription would have been deleted when the first consumer was closed.

          3. Your option (C) shouldn't then be needed

          4. You suggest "it would be worth pointing out (for any of the options) that the message still gets sent to the destination (topic) even if it doesn't get delivered to the subscription."

          Yes, definitely. More accurately, even if a message doesn't get added to one particular subscription on the topic, it will still be added to all other elegible subscriptions on the same topic.

          To answer Chris's comment of 13/Sep/12 09:51 PM:

          5. You write "There's no good reason I can see why the noLocal behavior should outlive the consumer (and the connection) in the durables case. After all, it is the subscription that is durable, not the consumer. And it is the consumer that has noLocal=true, not the subscription."

          That's quite an important point. However one could argue that the noLocal flag is really just another kind of message selector on the subscription (based on the client id that sent the message), and so continues to exist irrespective of whether there were any consumers active.

          However I think we ultimately have a slightly arbitrary decision here. noLocal is a distinctly obscure feature: it would be interested to know how much it gets used.

          We should also consider an option D, which is when the new createSharedConsumer method is used to create a shared non-durable subscription (which is a new feature of JMS 2.0), there is no noLocal option at all.

          Show
          Nigel Deakin added a comment - To Answer Chris's comment of 13/Sep/12 07:13 PM: 1. My option B was that if noLocal was true then once a connection has created a consumer on the subscription its messages would no longer be added to the subscription. This means they would not be delivered to any consumers. I think that was what you were asking for. As you say, if you have multiple consumers on the same subscription they are cooperating to process the messages so they should behave the same way. 2. You ask what would happen if "a shared consumer is created with noLocal=false, then a second is created on the same connection with noLocal=true (or vice versa). Which one wins?" If the first session still had a consumer on the shared non-durable subscription, then if a second connection or session tried to create a consumer on the same subscription with a different topic, message selector or noLocal value then they are trying to change the nature of the subscription, so it should throw an exception. Obviously if the first connection was closed before the second connection created the consumer then this would not be a problem, as the subscription would have been deleted when the first consumer was closed. 3. Your option (C) shouldn't then be needed 4. You suggest "it would be worth pointing out (for any of the options) that the message still gets sent to the destination (topic) even if it doesn't get delivered to the subscription." Yes, definitely. More accurately, even if a message doesn't get added to one particular subscription on the topic, it will still be added to all other elegible subscriptions on the same topic. To answer Chris's comment of 13/Sep/12 09:51 PM: 5. You write "There's no good reason I can see why the noLocal behavior should outlive the consumer (and the connection) in the durables case. After all, it is the subscription that is durable, not the consumer. And it is the consumer that has noLocal=true, not the subscription." That's quite an important point. However one could argue that the noLocal flag is really just another kind of message selector on the subscription (based on the client id that sent the message), and so continues to exist irrespective of whether there were any consumers active. However I think we ultimately have a slightly arbitrary decision here. noLocal is a distinctly obscure feature: it would be interested to know how much it gets used. We should also consider an option D, which is when the new createSharedConsumer method is used to create a shared non-durable subscription (which is a new feature of JMS 2.0), there is no noLocal option at all.

            People

            • Assignee:
              Nigel Deakin
              Reporter:
              Nigel Deakin
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: