JMS Server on a cluster stores all msgs posted to Distributed Queue on 1 server failure - Weblogic

This is a discussion on JMS Server on a cluster stores all msgs posted to Distributed Queue on 1 server failure - Weblogic ; Hi I have a Cluster setup with 2 managed servers srvr1 & srvr2 each running its own JMS server. Product version in use is Weblogic SP2. Configured a Distributed Queue and deployed it on both the JMS servers. Set the ...

+ Reply to Thread
Results 1 to 6 of 6

Thread: JMS Server on a cluster stores all msgs posted to Distributed Queue on 1 server failure

  1. JMS Server on a cluster stores all msgs posted to Distributed Queue on 1 server failure


    Hi

    I have a Cluster setup with 2 managed servers srvr1 & srvr2 each running its own
    JMS server.
    Product version in use is Weblogic SP2.
    Configured a Distributed Queue and deployed it on both the JMS servers.
    Set the load-balancing to round-Robin and Server-Affinity to False on the connection
    factory.

    The producer is an external client which posts 100 messages to the Distributed
    queue.
    Consumers are MDBs pinned to the respective physical queues on the respective
    JMS servers.
    Each server has its own store.

    Once 100 messages are posted to the Distributed Queue. The messages are distributed
    to
    the 2 Physical Destinations equally i.e 50-50 each.

    While the MDBs are processing the msgs, server srvr1 is brought down at a point
    when it has consumed only 10 of its 50 msgs.

    When the srvr1 is brought up again, i am finding that the store for that JMS server
    contains the 40 unprocessed msgs in its physical queue + the msgs on the other
    queue that were unprocessed at the time this server crashed.

    When srvr1 went down, srvr2 was running and it went on to process all its 50 msgs
    completly. Why would those processed msgs show up again when srvr1 is brought
    up ?

    Please provide some input !!

    Ravi.

  2. Re: JMS Server on a cluster stores all msgs posted to DistributedQueue on 1 server failure

    Seems more than odd. Are you sure that q1 got unprocessed
    messages from q2? How did you confirm?

    Perhaps q1 got messages its MDB already processed back again -
    an indication that the MDB is throwing Runtime exceptions
    or Errors, and the container is rolling back transactions.

    Raviprasad Athivilli wrote:
    > Hi
    >
    > I have a Cluster setup with 2 managed servers srvr1 & srvr2 each running its own
    > JMS server.
    > Product version in use is Weblogic SP2.
    > Configured a Distributed Queue and deployed it on both the JMS servers.
    > Set the load-balancing to round-Robin and Server-Affinity to False on the connection
    > factory.
    >
    > The producer is an external client which posts 100 messages to the Distributed
    > queue.
    > Consumers are MDBs pinned to the respective physical queues on the respective
    > JMS servers.
    > Each server has its own store.
    >
    > Once 100 messages are posted to the Distributed Queue. The messages are distributed
    > to
    > the 2 Physical Destinations equally i.e 50-50 each.
    >
    > While the MDBs are processing the msgs, server srvr1 is brought down at a point
    > when it has consumed only 10 of its 50 msgs.
    >
    > When the srvr1 is brought up again, i am finding that the store for that JMS server
    > contains the 40 unprocessed msgs in its physical queue + the msgs on the other
    > queue that were unprocessed at the time this server crashed.
    >
    > When srvr1 went down, srvr2 was running and it went on to process all its 50 msgs
    > completly. Why would those processed msgs show up again when srvr1 is brought
    > up ?
    >
    > Please provide some input !!
    >
    > Ravi.



  3. Re: JMS Server on a cluster stores all msgs posted to Distributed


    Tom Barnes wrote:
    >Seems more than odd. Are you sure that q1 got unprocessed
    >messages from q2? How did you confirm?
    >
    >Perhaps q1 got messages its MDB already processed back again -
    >an indication that the MDB is throwing Runtime exceptions
    >or Errors, and the container is rolling back transactions.
    >
    >Raviprasad Athivilli wrote:
    >> Hi
    >>
    >> I have a Cluster setup with 2 managed servers srvr1 & srvr2 each running

    >its own
    >> JMS server.
    >> Product version in use is Weblogic SP2.
    >> Configured a Distributed Queue and deployed it on both the JMS servers.
    >> Set the load-balancing to round-Robin and Server-Affinity to False

    >on the connection
    >> factory.
    >>
    >> The producer is an external client which posts 100 messages to the

    >Distributed
    >> queue.
    >> Consumers are MDBs pinned to the respective physical queues on the

    >respective
    >> JMS servers.
    >> Each server has its own store.
    >>
    >> Once 100 messages are posted to the Distributed Queue. The messages

    >are distributed
    >> to
    >> the 2 Physical Destinations equally i.e 50-50 each.
    >>
    >> While the MDBs are processing the msgs, server srvr1 is brought down

    >at a point
    >> when it has consumed only 10 of its 50 msgs.
    >>
    >> When the srvr1 is brought up again, i am finding that the store for

    >that JMS server
    >> contains the 40 unprocessed msgs in its physical queue + the msgs on

    >the other
    >> queue that were unprocessed at the time this server crashed.
    >>
    >> When srvr1 went down, srvr2 was running and it went on to process all

    >its 50 msgs
    >> completly. Why would those processed msgs show up again when srvr1

    >is brought
    >> up ?
    >>
    >> Please provide some input !!
    >>
    >> Ravi.

    >



  4. Re: JMS Server on a cluster stores all msgs posted to Distributed


    Tom Barnes wrote:
    >Seems more than odd. Are you sure that q1 got unprocessed
    >messages from q2? How did you confirm?
    >
    >Perhaps q1 got messages its MDB already processed back again -
    >an indication that the MDB is throwing Runtime exceptions
    >or Errors, and the container is rolling back transactions.
    >
    >Raviprasad Athivilli wrote:
    >> Hi
    >>
    >> I have a Cluster setup with 2 managed servers srvr1 & srvr2 each running

    >its own
    >> JMS server.
    >> Product version in use is Weblogic SP2.
    >> Configured a Distributed Queue and deployed it on both the JMS servers.
    >> Set the load-balancing to round-Robin and Server-Affinity to False

    >on the connection
    >> factory.
    >>
    >> The producer is an external client which posts 100 messages to the

    >Distributed
    >> queue.
    >> Consumers are MDBs pinned to the respective physical queues on the

    >respective
    >> JMS servers.
    >> Each server has its own store.
    >>
    >> Once 100 messages are posted to the Distributed Queue. The messages

    >are distributed
    >> to
    >> the 2 Physical Destinations equally i.e 50-50 each.
    >>
    >> While the MDBs are processing the msgs, server srvr1 is brought down

    >at a point
    >> when it has consumed only 10 of its 50 msgs.
    >>
    >> When the srvr1 is brought up again, i am finding that the store for

    >that JMS server
    >> contains the 40 unprocessed msgs in its physical queue + the msgs on

    >the other
    >> queue that were unprocessed at the time this server crashed.
    >>
    >> When srvr1 went down, srvr2 was running and it went on to process all

    >its 50 msgs
    >> completly. Why would those processed msgs show up again when srvr1

    >is brought
    >> up ?
    >>
    >> Please provide some input !!
    >>
    >> Ravi.

    >



  5. Re: JMS Server on a cluster stores all msgs posted to Distributed


    Tom

    I think you are right. When the server1 is brought up again, its picking up all
    the messages
    that were written to the store.

    But as it starts processing them, i get this exception on the server2 that has
    been running from the beginning.

    <040366> member "MyQueue@JMSServer-02" in "MyQueue"
    unable to accept connection from remote member "MyQueue@JMSServer-01" due to exception
    weblogic.jms.common.JMSException: can't find
    queue target jndi table for MyQueue@JMSServer-01 != MyQueue
    weblogic.jms.common.JMSException: can't find queue target jndi table for MyQueue@JMSServer-01
    != MyQueue
    at weblogic.jms.backend.BEDestination.setupSystemSubs cription(BEDestination.java:3466)
    at weblogic.jms.backend.BEManager.sessionAndTopicSubs criberCreate(BEManager.java:234)
    at weblogic.jms.backend.BEManager.invoke(BEManager.ja va:384)
    at weblogic.jms.dispatcher.Request.wrappedFiniteState Machine(Request.java:602)
    at weblogic.jms.dispatcher.DispatcherImpl.dispatchAsy nc(DispatcherImpl.java:152)
    at weblogic.jms.dispatcher.DispatcherImpl.dispatchAsy ncFuture(DispatcherImpl.java:396)
    at weblogic.jms.dispatcher.DispatcherImpl_WLSkel.invo ke(Unknown Source)
    at weblogic.rmi.internal.BasicServerRef.invoke(BasicS erverRef.java:362)
    at weblogic.rmi.internal.BasicServerRef$1.run(BasicSe rverRef.java:313)
    at weblogic.security.service.SecurityServiceManager.r unAs(SecurityServiceManager.java:821)
    at weblogic.rmi.internal.BasicServerRef.handleRequest (BasicServerRef.java:308)
    at weblogic.rmi.internal.BasicExecuteRequest.execute( BasicExecuteRequest.java:30)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThrea d.java:213)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.ja va:189)

    What does this mean ?

    Ravi.
    Tom Barnes wrote:
    >Seems more than odd. Are you sure that q1 got unprocessed
    >messages from q2? How did you confirm?
    >
    >Perhaps q1 got messages its MDB already processed back again -
    >an indication that the MDB is throwing Runtime exceptions
    >or Errors, and the container is rolling back transactions.
    >
    >Raviprasad Athivilli wrote:
    >> Hi
    >>
    >> I have a Cluster setup with 2 managed servers srvr1 & srvr2 each running

    >its own
    >> JMS server.
    >> Product version in use is Weblogic SP2.
    >> Configured a Distributed Queue and deployed it on both the JMS servers.
    >> Set the load-balancing to round-Robin and Server-Affinity to False

    >on the connection
    >> factory.
    >>
    >> The producer is an external client which posts 100 messages to the

    >Distributed
    >> queue.
    >> Consumers are MDBs pinned to the respective physical queues on the

    >respective
    >> JMS servers.
    >> Each server has its own store.
    >>
    >> Once 100 messages are posted to the Distributed Queue. The messages

    >are distributed
    >> to
    >> the 2 Physical Destinations equally i.e 50-50 each.
    >>
    >> While the MDBs are processing the msgs, server srvr1 is brought down

    >at a point
    >> when it has consumed only 10 of its 50 msgs.
    >>
    >> When the srvr1 is brought up again, i am finding that the store for

    >that JMS server
    >> contains the 40 unprocessed msgs in its physical queue + the msgs on

    >the other
    >> queue that were unprocessed at the time this server crashed.
    >>
    >> When srvr1 went down, srvr2 was running and it went on to process all

    >its 50 msgs
    >> completly. Why would those processed msgs show up again when srvr1

    >is brought
    >> up ?
    >>
    >> Please provide some input !!
    >>
    >> Ravi.

    >



  6. Re: JMS Server on a cluster stores all msgs posted to Distributed


    The log message is from the distributed destination
    "forwarders". In distributed queues, these forwarders
    automatically move messages from queues without
    consumers to queues that have them (queue
    forwarders are disabled by default). In distributed topics,
    the forwarders serve
    to replicate a published message from the local
    topic member to each of the remote topic members.



    "Raviprasad Athivilli" wrote:
    >
    >Tom
    >
    >I think you are right. When the server1 is brought up again, its picking
    >up all
    >the messages
    >that were written to the store.
    >
    >But as it starts processing them, i get this exception on the server2
    >that has
    >been running from the beginning.
    >
    > <040366> >Destination
    >member "MyQueue@JMSServer-02" in "MyQueue"
    >unable to accept connection from remote member "MyQueue@JMSServer-01"
    >due to exception
    >weblogic.jms.common.JMSException: can't find
    >queue target jndi table for MyQueue@JMSServer-01 != MyQueue
    >weblogic.jms.common.JMSException: can't find queue target jndi table
    >for MyQueue@JMSServer-01
    >!= MyQueue
    > at weblogic.jms.backend.BEDestination.setupSystemSubs cription(BEDestination.java:3466)
    > at weblogic.jms.backend.BEManager.sessionAndTopicSubs criberCreate(BEManager.java:234)
    > at weblogic.jms.backend.BEManager.invoke(BEManager.ja va:384)
    > at weblogic.jms.dispatcher.Request.wrappedFiniteState Machine(Request.java:602)
    > at weblogic.jms.dispatcher.DispatcherImpl.dispatchAsy nc(DispatcherImpl.java:152)
    > at weblogic.jms.dispatcher.DispatcherImpl.dispatchAsy ncFuture(DispatcherImpl.java:396)
    > at weblogic.jms.dispatcher.DispatcherImpl_WLSkel.invo ke(Unknown
    >Source)
    > at weblogic.rmi.internal.BasicServerRef.invoke(BasicS erverRef.java:362)
    > at weblogic.rmi.internal.BasicServerRef$1.run(BasicSe rverRef.java:313)
    > at weblogic.security.service.SecurityServiceManager.r unAs(SecurityServiceManager.java:821)
    > at weblogic.rmi.internal.BasicServerRef.handleRequest (BasicServerRef.java:308)
    > at weblogic.rmi.internal.BasicExecuteRequest.execute( BasicExecuteRequest.java:30)
    > at weblogic.kernel.ExecuteThread.execute(ExecuteThrea d.java:213)
    > at weblogic.kernel.ExecuteThread.run(ExecuteThread.ja va:189)
    >
    >What does this mean ?
    >
    >Ravi.
    >Tom Barnes wrote:
    >>Seems more than odd. Are you sure that q1 got unprocessed
    >>messages from q2? How did you confirm?
    >>
    >>Perhaps q1 got messages its MDB already processed back again -
    >>an indication that the MDB is throwing Runtime exceptions
    >>or Errors, and the container is rolling back transactions.
    >>
    >>Raviprasad Athivilli wrote:
    >>> Hi
    >>>
    >>> I have a Cluster setup with 2 managed servers srvr1 & srvr2 each running

    >>its own
    >>> JMS server.
    >>> Product version in use is Weblogic SP2.
    >>> Configured a Distributed Queue and deployed it on both the JMS servers.
    >>> Set the load-balancing to round-Robin and Server-Affinity to False

    >>on the connection
    >>> factory.
    >>>
    >>> The producer is an external client which posts 100 messages to the

    >>Distributed
    >>> queue.
    >>> Consumers are MDBs pinned to the respective physical queues on the

    >>respective
    >>> JMS servers.
    >>> Each server has its own store.
    >>>
    >>> Once 100 messages are posted to the Distributed Queue. The messages

    >>are distributed
    >>> to
    >>> the 2 Physical Destinations equally i.e 50-50 each.
    >>>
    >>> While the MDBs are processing the msgs, server srvr1 is brought down

    >>at a point
    >>> when it has consumed only 10 of its 50 msgs.
    >>>
    >>> When the srvr1 is brought up again, i am finding that the store for

    >>that JMS server
    >>> contains the 40 unprocessed msgs in its physical queue + the msgs

    >on
    >>the other
    >>> queue that were unprocessed at the time this server crashed.
    >>>
    >>> When srvr1 went down, srvr2 was running and it went on to process

    >all
    >>its 50 msgs
    >>> completly. Why would those processed msgs show up again when srvr1

    >>is brought
    >>> up ?
    >>>
    >>> Please provide some input !!
    >>>
    >>> Ravi.

    >>

    >



+ Reply to Thread