Environment: WLI 7.0x 4 managed servers in 2 boxes + 1 admin server
Message bridge source destination is IBM MQ 5.3 whose jndi name is
bound to weblogic naming context when each managed servers start up
via startup hookup.
target destination is WebLogic JMS Queue.
Using IBM MQ extended transation client, we are able to make the
message bridge QoS to "exactly-once". It works fine in developer box
(no cluster), UAT (cluster). In production, it works fine at first.
But recently we get exceptions related to message bridge sometimes.
I.E. sometimes it works and sometimes it fails. The exception can be
categoried into 5:
1st: javax.resource.spi.IllegalStateException: Connection is invalid
2nd: com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2072
3th: com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2042
(see below for detail stack trace)
4th: javax.resource.ResourceException:
javax.transaction.SystemException: start() failed on resource
'com.ibm.mq.MQXAResource': XAER_RMFAIL : Resource manager is
unavailable
5th: java.lang.Exception:
weblogic.transaction.internal.TimedOutException: Timed out
tx=28951:f4a0ce010710890e after 30 seconds

I'd appreciate any help!

####
'__weblogic_admin_rmi_queue'> <132:55f43da00710890e>
<200015> messages (java.lang.Exception: javax.resource.ResourceException: Error
creating consumer or receiving message
at weblogic.jms.adapter.JMSBaseConnection.throwResour ceException(JMSBaseConnection.java:1079)
at weblogic.jms.adapter.JMSBaseConnection.receiveInte rnal(JMSBaseConnection.java:766)
at weblogic.jms.adapter.JMSBaseConnection.receiveComm on(JMSBaseConnection.java:648)
at weblogic.jms.adapter.JMSBaseConnection.receive(JMS BaseConnection.java:803)
at weblogic.jms.adapter.JMSConnectionHandle.receive(J MSConnectionHandle.java:111)
at java.lang.reflect.Method.invoke(Native Method)
at weblogic.connector.common.internal.ConnectionWrapp er.invoke(ConnectionWrapper.java:101)
at $Proxy113.receive(Unknown Source)
at weblogic.jms.bridge.internal.MessagingBridge.proce ssMessages(MessagingBridge.java:1242)
at weblogic.jms.bridge.internal.MessagingBridge.begin Forwarding(MessagingBridge.java:872)
at weblogic.jms.bridge.internal.MessagingBridge.execu te(MessagingBridge.java:942)
at weblogic.kernel.ExecuteThread.execute(ExecuteThrea d.java:213)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.ja va:189)
-------------- Linked Exception ------------
javax.jms.JMSException: MQJMS2008: failed to open MQ queue
at com.ibm.mq.jms.MQQueueSession.getQueueOpenExceptio n(MQQueueSession.java:950)
at com.ibm.mq.jms.MQQueueSession.createReceiver(MQQue ueSession.java:331)
at com.ibm.mq.jms.MQQueueSession.createReceiver(MQQue ueSession.java:229)
at weblogic.jms.adapter.JMSBaseConnection.receiveInte rnal(JMSBaseConnection.java:710)
at weblogic.jms.adapter.JMSBaseConnection.receiveComm on(JMSBaseConnection.java:648)
at weblogic.jms.adapter.JMSBaseConnection.receive(JMS BaseConnection.java:803)
at weblogic.jms.adapter.JMSConnectionHandle.receive(J MSConnectionHandle.java:111)
at java.lang.reflect.Method.invoke(Native Method)
at weblogic.connector.common.internal.ConnectionWrapp er.invoke(ConnectionWrapper.java:101)
at $Proxy113.receive(Unknown Source)
at weblogic.jms.bridge.internal.MessagingBridge.proce ssMessages(MessagingBridge.java:1242)
at weblogic.jms.bridge.internal.MessagingBridge.begin Forwarding(MessagingBridge.java:872)
at weblogic.jms.bridge.internal.MessagingBridge.execu te(MessagingBridge.java:942)
at weblogic.kernel.ExecuteThread.execute(ExecuteThrea d.java:213)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.ja va:189)
-------------- Linked Exception 2 ------------
com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2042
at com.ibm.mq.MQQueueManager.accessQueue(MQQueueManag er.java:1527)
at com.ibm.mq.MQQueueManager.accessQueue(MQQueueManag er.java:1579)
at com.ibm.mq.jms.MQQueueSession.createReceiver(MQQue ueSession.java:328)
at com.ibm.mq.jms.MQQueueSession.createReceiver(MQQue ueSession.java:229)
at weblogic.jms.adapter.JMSBaseConnection.receiveInte rnal(JMSBaseConnection.java:710)
at weblogic.jms.adapter.JMSBaseConnection.receiveComm on(JMSBaseConnection.java:648)
at weblogic.jms.adapter.JMSBaseConnection.receive(JMS BaseConnection.java:803)
at weblogic.jms.adapter.JMSConnectionHandle.receive(J MSConnectionHandle.java:111)
at java.lang.reflect.Method.invoke(Native Method)
at weblogic.connector.common.internal.ConnectionWrapp er.invoke(ConnectionWrapper.java:101)
at $Proxy113.receive(Unknown Source)
at weblogic.jms.bridge.internal.MessagingBridge.proce ssMessages(MessagingBridge.java:1242)
at weblogic.jms.bridge.internal.MessagingBridge.begin Forwarding(MessagingBridge.java:872)
at weblogic.jms.bridge.internal.MessagingBridge.execu te(MessagingBridge.java:942)
at weblogic.kernel.ExecuteThread.execute(ExecuteThrea d.java:213)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.ja va:189)