[GLASSFISH-20998] JMS messages lose message properties when sent via JMSContext. Selector's don't work. Created: 01/Mar/14  Updated: 03/Jun/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: pranahata Assignee: David Zhao
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 4_0_1-reviewed

 Description   

When sending a message via JMSContext, none of the properties sent on the message arrive at the client, making jms message selectors unusable.

You can use the same test case as here, https://java.net/jira/browse/GLASSFISH-20973



 Comments   
Comment by Nigel Deakin [ 03/Mar/14 ]

Seems to work for me. Can you please paste the exact code you used to set the property?

I tried these two variants:

setting it directly on the Message:

try (JMSContext context = connectionFactory.createContext()){
   TextMessage textMessage = context.createTextMessage("Hello world");
   textMessage.setStringProperty("MyProperty", "Koala");
   context.createProducer().send(inboundQueue,m);
}

and setting it via the JMSProducer

try (JMSContext context = connectionFactory.createContext()){
   context.createProducer().setProperty("MyProperty", "Wombat").send(inboundQueue,"Hello world");
}

and checked the property was set on the received message using

String myProp = message.getStringProperty("MyProperty");

and confirmed that the expected property value was set.





[GLASSFISH-7279] receiveNoWait should poll broker Created: 09/Mar/09  Updated: 18/Jun/13

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: v2.1
Fix Version/s: future release

Type: Improvement Priority: Critical
Reporter: rahulbiswas Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issue Links:
Dependency
depends on JMS_SPEC-85 Clarify how Message.receiveNoWait() i... Open
Issuezilla Id: 7,279
Tags: bj-reviewed-out

 Description   

Need an option for the client to poll the broker for new messages, for the
receiveNoWait api implementation. Currently the impl returns null even if there
are messages on the broker. This scenario been described in the following closed
bug – https://glassfish.dev.java.net/issues/show_bug.cgi?id=3011

With this option the client should poll the broker for new messages and return a
message if available on broker. If none are available, it should not block for
new messages to arrive on the broker.



 Comments   
Comment by kumara [ 01/Sep/09 ]

Changing version from 9.1.1 to v2.1 to reflect new name/version.

Comment by Tom Mueller [ 06/Mar/12 ]

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.





[GLASSFISH-20934]  "org.osgi.framework.BundleException: Unable to acquire global lock for resolve" happened while executing "asadmin start-cluster" Created: 18/Dec/13  Updated: 18/Dec/13

Status: Open
Project: glassfish
Component/s: hk2, jms, jts, OSGi
Affects Version/s: 3.1.2, 3.1.2.2, 4.0
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: TangYong Assignee: paul_parkinson
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File GLASSFISH-20934.patch    

 Description   

While executing "asadmin start-cluster" in an environment with heavy-load, sometimes, the following exception happened in server.log

...
[#|2013-12-04T16:46:56.553+0900|SEVERE|||_ThreadID=68;_ThreadName=Recovery Helper Thread;|Exception in thread "Recovery Helper Thread" |#]

[#|2013-12-04T16:46:56.558+0900|SEVERE|||_ThreadID=68;_ThreadName=Recovery Helper Thread;|com.sun.enterprise.module.ResolveError: Failed to start Bundle Id [40] State [INSTALLED] [org.glassfish.main.connectors.inbound-runtime(Connectors Inbound Support):3.1.2.2-XXX-SNAPSHOT]
at org.jvnet.hk2.osgiadapter.OSGiModuleImpl.start(OSGiModuleImpl.java:177)
at org.jvnet.hk2.osgiadapter.OSGiModuleImpl$2$1$1.loadClass(OSGiModuleImpl.java:344)
at com.sun.hk2.component.LazyInhabitant.loadClass(LazyInhabitant.java:124)
at com.sun.hk2.component.LazyInhabitant.fetch(LazyInhabitant.java:111)
at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:135)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:78)
at org.jvnet.hk2.component.Habitat$5.get(Habitat.java:703)
at java.util.AbstractList$Itr.next(AbstractList.java:358)
at java.util.AbstractCollection.toArray(AbstractCollection.java:141)
at java.util.ArrayList.<init>(ArrayList.java:164)
at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.configure(ResourceRecoveryManagerImpl.java:339)
at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:231)
at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:331)
at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.postConstruct(ResourceRecoveryManagerImpl.java:106)
at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:131)
at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:91)
at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:82)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67)
at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:139)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:78)
at org.jvnet.hk2.component.Habitat.getByContract(Habitat.java:1050)
at com.sun.jts.jta.TransactionServiceProperties$RecoveryHelperThread.run(TransactionServiceProperties.java:358)
Caused by: org.osgi.framework.BundleException: Unable to acquire global lock for resolve.
at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:3832)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1868)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:944)
at org.jvnet.hk2.osgiadapter.OSGiModuleImpl.start(OSGiModuleImpl.java:169)
... 21 more

#]
...

Among the above, [40] is org.glassfish.main.connectors.inbound-runtime bundle.

This issue maybe also happened on 4.0 based on my source analyze.

About "Component/s", firstly, I selected HK2, OSGi and JMS. About why selecting JMS, I will say the reason in the following comment. secondly, I firstly assign to Sahoo to evaluate.



 Comments   
Comment by TangYong [ 18/Dec/13 ]

[My investigation]

In starting cluster process, there are two threads which all influence org.glassfish.main.connectors.inbound-runtime module's state. The two threads are:

1) Recovery Helper Thread
The thread will auto-start org.glassfish.main.connectors.inbound-runtime module while the following from org.glassfish.main.transaction.jts module is executed:

ResourceRecoveryManager recoveryManager = serviceLocator.getService(ResourceRecoveryManager.class)

By a series of calling, finally, executing logic will search the implementation or service of com.sun.enterprise.transaction.spi.RecoveryResourceHandler interface. And InboundRecoveryHandler class from org.glassfish.main.connectors.inbound-runtime module will be a candidate.

2) RunLevelController Thread
The thread comes from HK2. And the thread is produced because while gf kernel is starting, the following from com.sun.enterprise.v3.server.AppServerStartup will make effect,

if (!proceedTo(PostStartupRunLevel.VAL))

{ … }

Well, hk2 will filter all descriptors meeting run level equaling with PostStartupRunLevel.VAL(20) and then starting these modules. Because JmsProviderLifecycle class from org.glassfish.main.jms.core defines @RunLevel(value=PostStartupRunLevel.VAL, mode=RunLevel.RUNLEVEL_MODE_NON_VALIDATING), so it is a candidate. Deeply, while starting org.glassfish.main.jms.core, because org.glassfish.main.jms.core depends on org.glassfish.main.connectors.inbound-runtime module[1], firstly, OSGi runtime will resolve org.glassfish.main.connectors.inbound-runtime module.

[1]: only ActiveJmsResourceAdapter class depends on org.glassfish.main.connectors.inbound-runtime.

Based on the above analyze, if the two threads are in cross running, Recovery Helper Thread will be interrupted by felix because it can not obtain global lock, about the issue, pl. seeing [2].

Richard says:
"This is not necessarily a bug. That can happen if your bundle is holding a bundle lock and you try to acquire the global lock, which in this case the thread holds the bundle lock for the bundle it is trying to start and it needs the global lock to resolve it. If someone else already has the global lock and needs a bundle lock, then it will interrupt the other thread only holding the bundle lock. This avoids deadlocks."

[2]:http://mail-archives.apache.org/mod_mbox/felix-users/201102.mbox/%3C4D498058.7080104@ungoverned.org%3E

So, this is why I selected HK2, OSGi and JMS.

Comment by TangYong [ 18/Dec/13 ]

Deeply, I have some new finding as following:

Recovery Helper Thread is triggered by the following from com.sun.enterprise.v3.server.AppServerStartup,

if (!postStartupJob())

{ ... }

And RunLevelControllerThread running PostStartupRunLevel.VAL(20) is triggered by the following:

if (!proceedTo(PostStartupRunLevel.VAL))

{ ... }

Apparently, if postStartupJob has not truly been finished and RunLevelControllerThread running PostStartupRunLevel.VAL(20) starts to run, then, the issue or similar issues(module state changing) can happen.

Tang

Comment by TangYong [ 18/Dec/13 ]

A fixing way is delaying Recovery Helper Thread's running and after PostStartupRunLevel has been finished, by sending event, running Recovery Helper Thread.

about detailed fixing way, pl. seeing and confirming the patch.

Thanks
Tang

Comment by TangYong [ 18/Dec/13 ]

Adding JTS component and request JTS leader to confirm the patch and evaluate the issue.





[GLASSFISH-21289] JMS broker fails on concurrent writes to single topic when using distributed transactions. Created: 17/Jan/15  Updated: 11/Mar/15

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.1
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: jiggster Assignee: David Zhao
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Tested with glass fish-embedded on Windows 7 & Mac OSX 10.10.1


Tags: 4-0-b78

 Description   

When multiple threads write to single JMS topic (only tested with topic, but that might be true for the queues also), each in its own distributed transaction, the broker fails with an error like the one below:

Jan 17, 2015 10:13:51 PM com.sun.messaging.jms.ra.DirectXAResource validateAndSaveXidTransactionID
INFO: DXAR:start():Warning: XAResource with state COMPLETE received diff Xid for open txnId:switching transactionId:
DXAR  Xid=(GlobalTransactionID=[B@1d8256f1, BranchQualifier=[B@42963311) 
DXAR TXid=5598056773328087040
got   Xid=(GlobalTransactionID=[B@1c462fe0, BranchQualifier=[B@47c43f17) 
got  TXid=5598056773328088320
Jan 17, 2015 10:13:51 PM com.sun.messaging.jms.ra.DirectXAResource prepare
SEVERE: prepareTransaction (XA) on JMSService:jmsdirect failed for connectionId:5598056773326219776 due to Unknown JMSService server error ERROR: com.sun.messaging.jmq.jmsserver.util.BrokerException: Bad transaction state transition. Cannot perform operation PREPARE_TRANSACTION(56) (XAFlag=null) on a transaction in state STARTED(1).
Jan 17, 2015 10:13:51 PM com.sun.jts.CosTransactions.RegisteredResources distributePrepare
WARNING: JTS5031: Exception [java.lang.RuntimeException: javax.transaction.xa.XAException] on Resource [prepare] operation.
Jan 17, 2015 10:13:51 PM com.sun.messaging.jms.ra.DirectXAResource rollback
SEVERE: rollbackTransaction (XA) on JMSService:jmsdirect failed for connectionId:5598056773326219776:transactionId=5598056773328088320 due to Unknown JMSService server error ERROR: com.sun.messaging.jmq.jmsserver.util.BrokerException: Transaction XID mismatch 6A696767626F6F6B2E6C6F63616C2C7365727665722C503130302C013B0000004D8FBEF96A696767626F6F6B2E6C6F63616C2C7365727665722C50313030, expected 6A696767626F6F6B2E6C6F63616C2C7365727665722C503130302C013C0000004D8FBEF96A696767626F6F6B2E6C6F63616C2C7365727665722C50313030 for transaction 5598056773328088320
Jan 17, 2015 10:13:51 PM com.sun.jts.jtsxa.OTSResourceImpl rollback
WARNING: JTS5068: Unexpected error occurred in rollback
javax.transaction.xa.XAException
	at com.sun.messaging.jms.ra.DirectXAResource.rollback(DirectXAResource.java:738)
	at com.sun.messaging.jms.ra.DirectXAResource.rollback(DirectXAResource.java:689)
	at com.sun.jts.jta.TransactionState._rollback(TransactionState.java:202)
	at com.sun.jts.jta.TransactionState.rollback(TransactionState.java:180)
	at com.sun.jts.jtsxa.OTSResourceImpl.rollback(OTSResourceImpl.java:333)
	at com.sun.jts.CosTransactions.RegisteredResources.distributeRollback(RegisteredResources.java:1040)
	at com.sun.jts.CosTransactions.TopCoordinator.rollback(TopCoordinator.java:2291)
	at com.sun.jts.CosTransactions.CoordinatorTerm.commit(CoordinatorTerm.java:391)
	at com.sun.jts.CosTransactions.TerminatorImpl.commit(TerminatorImpl.java:231)
	at com.sun.jts.CosTransactions.CurrentImpl.commit(CurrentImpl.java:622)
	at com.sun.jts.jta.TransactionManagerImpl.commit(TransactionManagerImpl.java:331)
	at com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate.commitDistributedTransaction(JavaEETransactionManagerJTSDelegate.java:185)
	at com.sun.enterprise.transaction.JavaEETransactionManagerSimplified.commit(JavaEETransactionManagerSimplified.java:859)
	at com.sun.enterprise.transaction.UserTransactionImpl.commit(UserTransactionImpl.java:212)
	at com.ibm.jbatch.container.transaction.impl.JTAUserTransactionAdapter.commit(JTAUserTransactionAdapter.java:91)
	at com.ibm.jbatch.container.impl.ChunkStepControllerImpl.invokeChunk(ChunkStepControllerImpl.java:620)
	at com.ibm.jbatch.container.impl.ChunkStepControllerImpl.invokeCoreStep(ChunkStepControllerImpl.java:684)
	at com.ibm.jbatch.container.impl.BaseStepControllerImpl.execute(BaseStepControllerImpl.java:144)
	at com.ibm.jbatch.container.impl.ExecutionTransitioner.doExecutionLoop(ExecutionTransitioner.java:112)
	at com.ibm.jbatch.container.impl.JobThreadRootControllerImpl.originateExecutionOnThread(JobThreadRootControllerImpl.java:110)
	at com.ibm.jbatch.container.util.BatchWorkUnit.run(BatchWorkUnit.java:80)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.glassfish.enterprise.concurrent.internal.ManagedFutureTask.run(ManagedFutureTask.java:141)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
	at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
Caused by: com.sun.messaging.jmq.jmsservice.JMSServiceException: rollbackTransaction: rollback transaction failed. Connection ID: 5598056773326219776, Transaction ID: 5598056773328088320, XID: (Available at FINE log level) Caused by:com.sun.messaging.jmq.jmsserver.util.BrokerException: Transaction XID mismatch 6A696767626F6F6B2E6C6F63616C2C7365727665722C503130302C013B0000004D8FBEF96A696767626F6F6B2E6C6F63616C2C7365727665722C50313030, expected 6A696767626F6F6B2E6C6F63616C2C7365727665722C503130302C013C0000004D8FBEF96A696767626F6F6B2E6C6F63616C2C7365727665722C50313030 for transaction 5598056773328088320
	at com.sun.messaging.jmq.jmsserver.service.imq.JMSServiceImpl.rollbackTransaction(JMSServiceImpl.java:1718)
	at com.sun.messaging.jms.ra.DirectXAResource.rollback(DirectXAResource.java:714)
	... 27 more
Caused by: com.sun.messaging.jmq.jmsserver.util.BrokerException: Transaction XID mismatch 6A696767626F6F6B2E6C6F63616C2C7365727665722C503130302C013B0000004D8FBEF96A696767626F6F6B2E6C6F63616C2C7365727665722C50313030, expected 6A696767626F6F6B2E6C6F63616C2C7365727665722C503130302C013C0000004D8FBEF96A696767626F6F6B2E6C6F63616C2C7365727665722C50313030 for transaction 5598056773328088320
	at com.sun.messaging.jmq.jmsserver.data.protocol.ProtocolImpl.rollbackTransaction(ProtocolImpl.java:921)
	at com.sun.messaging.jmq.jmsserver.service.imq.JMSServiceImpl.rollbackTransaction(JMSServiceImpl.java:1706)
	... 28 more

I've created a test case that reproduces the issue quite repeatedly - it's available on github.

The test case consists of a batch job that contains a single chunk-style step with partition mapper (step's partitioned as the problem occurs in a concurrent environment). ItemReader creates random number (between MIN and MAX as defined in SimplePartitionMapper) of entity instances of type Subscriber. ItemProcessor does nothing, but sleeps for 50 ms. Item writer persists entities created by the reader and then publishes the collection of items to JMS topic (in a single distributed transaction) and here's where the problem occurs. Will try to provide the more detailed description of the test case in the README.md file on github.



 Comments   
Comment by stephanbauer7 [ 11/Mar/15 ]

Hi,
I have the same problem with Queues in Glassfish 4.1. I am also using XA-Transactions.

Information:   11 Mrz 2015 10:55:53,039 [p: thread-pool-1; w: 3] DEBUG org.apache.ibatis.logging.jdbc.BaseJdbcLogger:132 - ==>  Preparing: delete from ACT_RU_EVENT_SUBSCR where ID_ = ? and REV_ = ?
Information:   11 Mrz 2015 10:55:53,039 [pool-22-thread-3] DEBUG org.activiti.engine.impl.db.DbSqlSession:828 - executing: delete JobEntity [id=42532]
Information:   11 Mrz 2015 10:55:53,039 [pool-22-thread-3] DEBUG org.apache.ibatis.logging.jdbc.BaseJdbcLogger:132 - ooo Using Connection [com.sun.gjc.spi.jdbc40.ConnectionWrapper40@4e70cf77]
Schwerwiegend:   commitTransaction (XA) on JMSService:jmsdirect failed for connectionId:6980944495603333120 and onePhase:true due to Unknown JMSService server error ERROR: com.sun.messaging.jmq.jmsserver.util.BrokerException: Transaction XID mismatch 6D75686861676C2C7365727665722C50333730302C000C000000660039086D75686861676C2C7365727665722C5033373030, expected 6D75686861676C2C7365727665722C50333730302C000B000000660039086D75686861676C2C7365727665722C5033373030 for transaction 6980944495768780544
Information:   11 Mrz 2015 10:55:53,039 [p: thread-pool-1; w: 3] DEBUG org.apache.ibatis.logging.jdbc.BaseJdbcLogger:132 - ==> Parameters: 40068(String), 1(Integer)
Information:   11 Mrz 2015 10:55:53,039 [pool-22-thread-3] DEBUG org.apache.ibatis.logging.jdbc.BaseJdbcLogger:132 - ==>  Preparing: delete from ACT_RU_JOB where ID_ = ? and REV_ = ?
Schwerwiegend:   rollbackTransaction (XA) on JMSService:jmsdirect failed for connectionId:6980944495603333120:transactionId=6980944495768780544 due to Unknown JMSService server error ERROR: com.sun.messaging.jmq.jmsserver.util.BrokerException: Transaction XID mismatch 6D75686861676C2C7365727665722C50333730302C000C000000660039086D75686861676C2C7365727665722C5033373030, expected 6D75686861676C2C7365727665722C50333730302C000B000000660039086D75686861676C2C7365727665722C5033373030 for transaction 6980944495768780544
Information:   11 Mrz 2015 10:55:53,040 [p: thread-pool-1; w: 3] DEBUG org.apache.ibatis.logging.jdbc.BaseJdbcLogger:132 - <==    Updates: 1
Comment by amyk [ 11/Mar/15 ]

Assuming the test case follows the JMS and Java EE spec (haven't looked at it myself) in using transaction, JMS Session, JMSContext etc., I suspect this is likely an issue with jms-service DIRECT (or EMBEDDED) mode for it uses a completely different path in JMSRA to interact with broker. I'd suggest first check the test case to ensure spec compliance, then try jms-service LOCAL mode.





[GLASSFISH-20987] Setting imq.hostname in broker breaks JMX access Created: 17/Feb/14  Updated: 23/Apr/15

Status: Open
Project: glassfish
Component/s: amx, jms
Affects Version/s: 4.0
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: electricsam Assignee: prasads
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux, jdk 1.7.0_45


Tags: javaee_ri_target

 Description   

I am trying to use an enhanced LOCAL broker cluster in my setup and would like to set up some monitoring. However, I was unable to connect to the MBeans though jconsole.

I used the following command to get the uri for the JMX connection:

imqcmd -b <hostname>:27676 list jmx

which produced:

service:jmx:rmi://<hostname>/jndi/rmi://<hostname>:1099/<hostname>/7676/jmxrmi

When I tried to use that in jconsole, I received a connection error.

To try to narrow the problem down, I set up a remote broker and configured it with an HA configuration and started removing properties. As soon as I removed the imq.hostname property, I was able to connect.






[GLASSFISH-13691] Provide way to disable JMS via command line Created: 29/Sep/10  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1
Fix Version/s: 4.1

Type: Improvement Priority: Major
Reporter: Tom Mueller Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 13,691
Tags: bj-reviewed-t1

 Description   

The JMS functionality in the GlassFish server can be disabled by editing the
domain.xml to have an empty or missing <jms-service> element. However, there
isn't any way to do this from the asadmin command line. This RFE is a request for
being able to do this for asadmin.



 Comments   
Comment by bqin [ 19/Oct/12 ]

This is not a must for the release 4.0, defer it to 4.0.1.





[GLASSFISH-12564] Cannot set null string property on Object Message Created: 07/Jul/10  Updated: 19/Sep/14

Status: Reopened
Project: glassfish
Component/s: jms
Affects Version/s: v3.0.1
Fix Version/s: 4.1

Type: Bug Priority: Major
Reporter: exabrial Assignee: Nigel Deakin
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issue Links:
Dependency
depends on MQ-165 Cannot set null message property Open
Issuezilla Id: 12,564
Tags: 3_1-exclude

 Description   

glassfish v3.0.1
sun jdk 1.6.0_20
windows xp 32bit

The JMS Message interface javadoc states that null is valid value for a string
property on a message.

If you invoke this code in a stateless ejb, you receive the stack trace below

Connection connection = connectionFactory.createConnection();
Session session = connection.createSession(true, Session.AUTO_ACKNOWLEDGE);
MessageProducer producer = session.createProducer(destination);
ObjectMessage message = session.createObjectMessage();
message.setObject(new Object());
message.setStringProperty("aStringProperty", null);

Caused by: java.lang.NullPointerException
at
com.sun.messaging.jms.ra.DirectPacket._checkAndSetProperty(DirectPacket.java:157
9)
at
com.sun.messaging.jms.ra.DirectPacket.setStringProperty(DirectPacket.java:1390)



 Comments   
Comment by Satish Kumar [ 06/Oct/10 ]

Setting target milestone and reassigning this to Nigel ..

Comment by Nigel Deakin [ 11/Oct/10 ]

Updating target milestone to M5 to confirm that this is already fixed.

Comment by Nigel Deakin [ 12/Oct/10 ]

Oops, ignore previous comment added to the wrong issue in error. Setting target
milestone back to M7.

Comment by Nigel Deakin [ 12/Oct/10 ]

In GlassFish 3.1 the error is

SEVERE: Error processing message
javax.jms.MessageFormatException: [C4040]: Invalid ObjectProperty type for
property aStringProperty
at
com.sun.messaging.jmq.jmsclient.MessageImpl.checkAndSetProperty(MessageImpl.java:818)
at
com.sun.messaging.jmq.jmsclient.MessageImpl.setStringProperty(MessageImpl.java:2041)

Comment by Nigel Deakin [ 12/Oct/10 ]

(previous stack trace was for LOCAL brokers),

And in GlassFish 3.1 the stack trace is

Caused by: java.lang.NullPointerException
at
com.sun.messaging.jms.ra.DirectPacket._checkAndSetProperty(DirectPacket.java:1581)
at
com.sun.messaging.jms.ra.DirectPacket.setStringProperty(DirectPacket.java:1392)

which matches what the submitted reported.

Comment by Nigel Deakin [ 12/Oct/10 ]

I've read the JMS 1.1 specification and the JavaDoc for javax.jms.Message, and
have not seen anything which suggests setting a String property can be set to
null. (Feel free to argue).

The behaviour when using a LOCAL broker is correct, which is to throw a
MessageFormatException.

The behaviour when using a EMBEDDED broker is incorrect. A NullPointerException
should not be thrown; a MessageFormatException should be thrown instead. I will
make this change.

Comment by Nigel Deakin [ 12/Oct/10 ]

The exception that is thrown when an embedded broker is used has been changed to:

javax.jms.MessageFormatException: MQJMSRA_DM4001:
setStringProperty():name=aStringProperty:value=null:Bad property value: null

Comment by Nigel Deakin [ 18/Oct/10 ]

Section 3.12 of the JMS spec "Provider Implementations of JMS Message
Interfaces" states:

"The JMS message interfaces provide write/set methods for setting object
values in a message body and message properties. All of these methods must
be implemented to copy their input objects into the message. The value of an
input object is allowed to be null and will return null when accessed."

Reopening this issue and setting target milestone to M7.

Comment by Nigel Deakin [ 26/Oct/10 ]

This does seem a valid complaint, though as it's a spec misinterpretation rather
than a bug, and MQ has almost certainly had this behaviour for many years,
fixing it can't be considered a priority for 3.1.

I'm leaving this as a P3 priority, but deferring it until 3.2 (MQ 4.6).

Comment by vladchuk [ 11/Dec/11 ]

Why is this not being addressed in 3.1.2? It seems that 3.2 has been abandoned (superseded by 4.0) and 4.0 is far away.
Please consider fixing it in 3.1.2.

Comment by exabrial [ 09/Jan/12 ]

any chance this could be fixed in 3.1.2? I can't imagine this is a difficult fix.

Comment by Nigel Deakin [ 03/May/12 ]

I've logged MQ-165 and made this issue dependent on it.

Comment by Nigel Deakin [ 15/Feb/13 ]

In accordance with the project triage guidelines this is not needed for 4.0 and so has been deferred until 4.0.1. Setting "fix version" accordingly.





[GLASSFISH-10775] JMSJCA application cannot access singleton objects created by JMSRA Created: 03/Nov/09  Updated: 06/Mar/12

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: V3
Fix Version/s: not determined

Type: Bug Priority: Major
Reporter: Nigel Deakin Assignee: Nigel Deakin
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: PC


Attachments: Zip Archive EnterpriseApplicationJMSJCA-ejb.zip     Zip Archive EnterpriseApplicationJMSJCA-war.zip     File sun-jms-adapter.rar    
Issuezilla Id: 10,775
Status Whiteboard:

v3_exclude

Tags: 3_1-exclude

 Description   

It appears that a deployed application using the JMSJCA RA cannot access
singleton objects created by the JMSRA system RA.

This can be seen if you deploy an application that uses JMSJCA to connect to an
embedded MQ broker using MQ's direct mode client rather than the longstanding
TCP mode client.

If you do this, the application fails because it thinks that the embedded broker
is not running. This is because the singleton class used for the two to connect
is being loaded by different class loaders, leading to their being two instances
of this singleton object.

This is a regression from Glassfish 2.1.1, in which JMSJCA+direct mode works
just fine.

I have written a simple EJB application to demonstrate the issue.

1. Deploy the attached JMSJCA resource adapter.

Please deploy using --libraries
$GLASSFISH/glassfish/lib/install/applications/jmsra/imqjmsra.jar,$GLASSFISH/glassfish/lib/install/applications/jmsra/imqbroker.jar
where $GLASSFISH is the root of your glassfish installation (the top-level
directory containing glassfish and mq)

2. Deploy the attached EnterpriseApplicationJMSJCA-ejb.zip

This is a Stateless session bean with one business method, doCase(int x). Call
this with x=1 to execute the test case. It does this:
Connection connection1 =
enterpriseApplicationJMSJCAConnectionFactory.createConnection();
Session session1 = connection1.createSession(false, Session.AUTO_ACKNOWLEDGE);

MessageProducer messageProducer =
session1.createProducer(enterpriseApplicationJMSJCAQueue);
messageProducer.send(createJMSMessageForjmsTestQueue(session1, "Hello"));

connection1.close();

where the connection factory and destination are defined in sun-resources.xml

3. Deploy the attached EnterpriseApplicationJMSJCA-war.zip

This is a simple servlet which invokes the session bean.

4. Start the embedded broker

Make sure the JMSRA is has been lazily started by invoking "asadmin jms-ping"

5. Run the test case

To run the test case, install the RA and use Netbeans to deploy both
applications (Netbeans should create the required resources).

Then use a web browser to goto
http://localhost:8080/EnterpriseApplicationJMSJCA-war/TestServlet?param=1

Look at the server log. This will show the errors:

WARNING: [I500]: Caught JVM Exception: java.lang.RuntimeException: Direct broker
not initialized for this client runtime.
WARNING: [C4003]: Error occurred on connection creation [mq://localhost/direct].

  • cause: javax.jms.JMSException: Direct broker not initialized for this client
    runtime.

5. More information

I believe the reason for this error is that I am getting duplicate classes
com.sun.messaging.jmq.jmsclient.runtime.impl.ClientRuntimeImpl and
com.sun.messaging.jmq.jmsclient.runtime.impl.ClientRuntime
which mean I get duplicate copies of each class's singleton instance.

These classes can be found in the following jars:

$GLASSFISH/mq/lib/imq.jar
$GLASSFISH/glassfish/lib/install/applications/jmsra/imqjmsra.jar

(though I got the same error even if I removed the relevant classes from
imqjmsra.jar and copied imq.jar into $GLASSFISH/glassfish/lib/install/applications)

One instance is created when JMSRA is started and is loaded by EJBClassLoader.

The other instance is created when the application is run, and is loaded by
com.sun.enterprise.v3.server.AppLibClassLoaderServiceImpl$URLClassFinder

6. Workaround attemts

I had several attemots at working round this issue:

6A: I left sun-jms-adapter deployed using the --libraries option and added the
same jars to "classpath-suffix". This made no difference, the tests still failed
with the duplicate class problem. (I did the same for classpath-prefix).

6B: I then removed the --libraries option and repeated the test. Now the test
failed with a class not found error: sun-jms-adapter could not find the MQ
client classes, despite the jars being included in in the jars added by
classpath-suffix.

6C: If I simply add the jars to domains/domain1/lib everything passes fine.
Obviously this is not a solution, but it shows how this is a classpath/class
loader issue.



 Comments   
Comment by Nigel Deakin [ 04/Nov/09 ]

Created an attachment (id=3754)
sun-jms-adapter.rar

Comment by Nigel Deakin [ 04/Nov/09 ]

Created an attachment (id=3755)
EnterpriseApplicationJMSJCA-ejb.zip

Comment by Nigel Deakin [ 04/Nov/09 ]

Created an attachment (id=3756)
EnterpriseApplicationJMSJCA-war.zip

Comment by jthoennes [ 04/Nov/09 ]

...

Comment by Jagadish [ 06/Nov/09 ]

After evaluating the issue by having discussions over email, we do not see an
issue with GlassFish's connector classloading behavior.

Following changes can be made in the resource-adapter so as to achieve the
requirement of sharing same class across resource-adapters.

1) Generate a library so that it can be shared across resource-adapters.
2) Use Installed libraries support provided in V3.
ie., deploy the .rar with --libraries support or
via EXTENSION_LIST in MANIFEST.MF of .rar
3) Use EXTENSION_LIST in MANIFEST.MF or jmsra to refer the the library that need
to be shared.

I have provided installed libraries support (EXTENSION_LIST) for system-resource
adapters now ( issue 10861 ).

Transferring to Nigel for further investigation.

For a sample, refer the following tests under :
https://svn.dev.java.net/svn/glassfish-svn/trunk/v2/appserv-tests/devtests/connector/v3

<ant dir="installed-libraries" target="all-ext"/>
<ant dir="installed-libraries" target="all--libraries"/>
<ant dir="installed-libraries-embedded" target="all"/>
<ant dir="installed-libraries-embedded" target="all-ext"/>

Refer :
https://svn.dev.java.net/svn/glassfish-svn/trunk/v2/appserv-tests/devtests/connector/README.txt
for general instructions.

Comment by Nigel Deakin [ 10/Nov/09 ]

Adding v3_exclude

Comment by kumara [ 07/Dec/09 ]

Setting target release for unresolved issues submitted on v3 release to the next release. Not changing
issues submitted on v2.x release because they might not apply to v3.next release.

Comment by Nigel Deakin [ 04/Oct/10 ]

Setting keyword 3.1-exclude.

Although this is a genuine issue, it doesn't need to be fixed for 3.1 because
the use of JMSJCA with direct mode was never formally supported (though it would
work with Glassfish 3 if it weren't for this issue), and since JMSJCA is in
sustaining mode this can't be considered a priority.

Comment by Tom Mueller [ 06/Mar/12 ]

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.





[GLASSFISH-15558] Caching JMS session in a session bean causes errors when invoked by a MDB when under load Created: 13/Jan/11  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1_b37
Fix Version/s: 4.1

Type: Bug Priority: Major
Reporter: Nigel Deakin Assignee: Nigel Deakin
Resolution: Unresolved Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File ServerLog.odt     Zip Archive TransactionTests.zip    
Tags: 3_1-exclude, 3_1-release-note-added, 3_1-release-notes, 3_1_1-scrubbed, 3_1_2-exclude

 Description   

This issue was originally reported by a user on the GlassFish developer list. Here is the thread:
http://www.java.net/forum/topic/glassfish/glassfish/jms-and-transaction-issue

The issue can be summarised as follows: a MDB consumes a message from an inbound queue, updates a database then invokes a session bean which sends a message to an outbound queue. If 40 messages are placed on the inbound queue then we see a variety of messages in the server log (see that thread), including this one:

625+0000|SEVERE|glassfish3.1|javax.resourceadapter.mqjmsra.outbound.connection|_ThreadID=36;_ThreadName=Thread-1;|commitTransaction (XA) on JMSService:jmsdirect failed for connectionId:950872495901869318 and onePhase:false due to Unknown JMSService server error ERROR: com.sun.messaging.jmq.jmsserver.util.BrokerException: Bad transaction state transition. Cannot perform operation COMMIT_TRANSACTION(46) (XAFlag=0x0:TMNOFLAGS) on a transaction in state COMPLETE(4).|#]

The error occurs ONLY if the session bean caches its JMS connection, session and producer in a field of the bean. This is valid, though it is contrary to the conventional practice which is to create the connection, use it, and close the connection every time the session bean is invoked. If these objects are not cached then this bug is NOT seen. This bug therefore has a workaround.

The issue can be reproduced using JMS only (i.e. not using a database), though to see exactly the same errors as the user reported it is necessary to force the use of two-phase commits.

A simple NetBeans application is attached which demonstrates the issue. This consists of a Enterprise Application "TransactionTests" which is composed of a ejb application "TransactionTests-ejb" and a web application "TransactionTests-war".

Steps to reproduce:

1. Install the latest version of GlassFish 3.1 (I used build 37)
2. Before starting GlassFish, edit domain.xml to set the JVM option -Dimq.jmsra.isSameRMAllowed=false . This is needed to force two-phase transactions to be used. (If this is not done the application will still fail but you will get different errors).
3. Use NetBeans to build the application (which is an ear cotaining an ejb and a web app) and deploy it in GlassFish.
4. Visit http://localhost:8080/TransactionTests-war/ and click on "Run MDB Test 1". This causes a servlet to send 40 messages to the inbound queue.
5. Inspect the server log for errors



 Comments   
Comment by Nigel Deakin [ 13/Jan/11 ]

The attached file ServerLog.odt is an extract from the server log, which includes logging in DirectXAResource and in the application.

Note particularly Thread 36 (highlighted in green) and Thread 51 (highlighted in red). This suggests that the session bean instance used by thread 36 was reused by thread 51 after the business method returned but before the MDB returned and the transaction was committed. This meant that the same JMS session object was being used by two threads at the same time, which caused the error.

(Full disclosure: to create this logging a modified version of MQ was used with all use to System.out() in DirectXAResource changed to use JDK logging. This was necessary to ensure that such log messages were reported using the correct thread)

Comment by Nigel Deakin [ 14/Jan/11 ]

This behaviour is also seen in GlassFish 2.1.1, so this is not a regression. There's also a workaround (and the workaround is generally considered better practice than the problem case). So this bug doesn't need to be fixed now, so setting the 3_1-exclude tag.

Comment by Nigel Deakin [ 14/Jan/11 ]

Have created documentation bug
http://java.net/jira/browse/GLASSFISH-15579
to record this in the release note for 3.1.

Comment by Paul Davies [ 14/Jan/11 ]

For the GlassFish 3.1 release notes add the following information:

A stateless session bean should not save JMS connections or sessions in fields of the bean. Applications that do so may encounter errors.

To avoid this issue, if a stateless session bean's business method requires the use of a JMS connection and session then the business method should create the JMS connection and session, use it to send or receive messages, and then close the connection and session before returning. This is GlassFish issue 15558.

Comment by theodor.richard [ 14/Jan/11 ]

I'm the user who initially reported this issue on the mailing list. A problem with not caching the connection is that the maximum number of connections is reached quickly. I'm seeing the following exceptions in the log when sending 50 messages in a for loop, i.e. the method that acquires and releases the JMS connection is invoked 50 times in a row:

com.sun.messaging.jms.JMSException: MQRA:DCF:allocation failure:createConnection:Error in allocating a connection. Cause: In-use connections equal max-pool-size and expired max-wait-time. Cannot allocate more connections.

My max connection pool size has the default size of 32.

Comment by Nigel Deakin [ 17/Jan/11 ]

@theodor.richard: If you believe that managed connections are not being returned correctly to the pool (and this isn't because your pool simply isn't big enough), then please log this as a separate issue or raise it on the user list. Please keep this issue for discussions of the effect of caching the connection, session and producer.

Comment by Nigel Deakin [ 25/Jan/11 ]

Analysis of the test case shows that the cause of the problem is that the container is reusing the session bean instance (and hence the connection's XAResource instance) after the business method has returned but before the transaction has been committed.

It is legal for the container to reuse the stateless session bean instance before the transaction has been committed: the EJB spec, section 4.7 "Stateless Session Beans" states that "the container may interleave requests from multiple transactions to the same instance".

However doing so causes errors in the JMSRA resource adapter, because it is designed on the basis that the same XAResource instance is used for start, end, prepare and commit and that the instance will not be reused until the transaction is committed or rolled back.

That is a breach of the JCA 1.5 spec, which states in section 7.3.2.1 "Implementation" that "A transaction manager can use any XAResource instance (if it refers to the proper resource manager instance) to initiate transaction completion. The XAResource instance used during the transaction completion process need not be the one initially enlisted with the transaction manager for this transaction"

This has been logged as internal (bugs.sun.com) bug 7014537.

Comment by jthoennes [ 14/Apr/11 ]

Hello Nigel,

as we heavily use that kind of scenario, I would like to ask whether this issue will be fixed for 3.1.1
without raising a service request.

A quick answer is highly appreciated.

Thanks, Jörg

Comment by Nigel Deakin [ 14/Apr/11 ]

This is currently scheduled for 3.2, though, as always, I can't make commitments as to the contents of future releases.

If you have a support licence and this issue is causing a problem them please contact your support representative (and let me know you've done so) since this would definitely affect the priority we give to fixing it.

Nigel

Comment by jthoennes [ 14/Apr/11 ]

In reply to comment #9:
> If you have a support licence and this issue is causing a problem them please
> contact your support representative (and let me know you've done so) since this
> would definitely affect the priority we give to fixing it.

Thanks, Nigel. Yes, we have a support contract. What do you need if I file a service request on My Oracle Support (MOS).
Do you have access to the service requests submitted?

Cheers, Jörg

Comment by Scott Fordin [ 15/Apr/11 ]

Added issue to 3.1 Release Notes.

Comment by Nazrul [ 21/Apr/11 ]

It would be good to take a look at this issue for 3.1.1

Comment by Nigel Deakin [ 03/May/11 ]

@jthoennes - Yes, please file an issue with Oracle support as you suggest. There is a separate engineering team to resolve customer issues, so raising it with support increases the resources available to address this issue.

Comment by Nigel Deakin [ 03/May/11 ]

I have reviewed this bug for 3.1.1 and decided not to fix it in that version for the following reasons:

  • This bug is in older versions of GlassFish (including GlassFish 2.1.1) and so is not a regression
  • There is a workaround (see earlier comment)
  • The fix would require significant changes to the XAResource implementation classes in the JMSRA resource adapter. In addition to the work involved it would require a lot of testing to be sure that it does not introduce a regression. 3.2 will have much more testing than 3.1.1 and so, given that this is an old bug which has a workaround, I would like to defer fixing this bug until 3.2 so it can be properly tested.

Removing the 3_1_1-review tag.

@jthoennes - note that if you raise this issue with Oracle support this will still be reviewed by Oracle sustaining.

Comment by jthoennes [ 27/May/11 ]

Filed Oracle Service Request "SR 3-3705874175: Resolve GLASSFISH-15558 for Glassfish 3.1.1" for this issue.

Comment by marina vatkina [ 16/Nov/11 ]

Re EJB container behavior: In our current implementation, bean instances are returned to pool at the end of method invocation. If we were to to delay it till the termination of tx, we would need more instances because transaction can last much longer than a single method invocation.

Comment by Nigel Deakin [ 14/Dec/11 ]

Adding 3_1_2-exclude tag. Excluding from 3.1.2 for the same reason it was excluded from 3.1.1 (see my comment above).





[GLASSFISH-15424] [BigApps] [STRESS] ~17 occurences of "EOFException" warnings coming from JMS Created: 03/Jan/11  Updated: 19/Sep/14  Due: 18/Jan/11

Status: Reopened
Project: glassfish
Component/s: jms
Affects Version/s: 3.1_b34
Fix Version/s: 4.1

Type: Bug Priority: Major
Reporter: varunrupela Assignee: Nigel Deakin
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File eof-issue.log     File server.log-instance101-24x1-gf-b37    
Issue Links:
Dependency
blocks GLASSFISH-15423 [STRESS] [BigApps] [Umbrella-Issue] 2... Closed
Tags: 3_1-exclude, 3_1-release-note-added, 3_1-release-notes, 3_1_1-scrubbed, 3_1_2-exclude, 3_1_2-release-note-added, 3_1_2-release-notes

 Description   

See parent issue 15423 for details of the BigApps run that causes this issue to appear in the server logs. A server log that shows the issue has been attached.



 Comments   
Comment by Satish Kumar [ 04/Jan/11 ]

This is a bug in MQ. See http://java.net/jira/browse/MQ-72 for the corresponding MQ issue. Amy Kang is currently working on it and based on her feedback, a fix for this issue will be a part of the next MQ integration ...

Comment by varunrupela [ 04/Jan/11 ]

Satish,

This is a different issue from the MQ-72 issue. The log is attached to the issue. We need to check on the cause for the log.

Comment by Satish Kumar [ 04/Jan/11 ]

As I had mentioned in my earlier comment, I suspect this issue may be caused due to http://java.net/jira/browse/MQ-72.

Lowering the priority of this issue to Minor as discussed with Varun. We will need to have a re-look at this once MQ-72 is fixed and then decide if any further action is required here or if we can close this issue.

Comment by Satish Kumar [ 05/Jan/11 ]

Bumping-up the priority to Major based on Nazrul's feedback.

We plan to leave this issue open until the fix for MQ 72 has been integrated and the stress tests have been run again and observe if the exceptions are reappearing in the new test run.

Comment by Nazrul [ 10/Jan/11 ]

Please confirm if MQ-72 resolved this problem.

Comment by varunrupela [ 12/Jan/11 ]

The issue continues to exist in build 37.

Comment by varunrupela [ 13/Jan/11 ]

Added one of the instance's server log after a 24x1 run of the same scenario.

Comment by Satish Kumar [ 13/Jan/11 ]

Assigning this issue to Nigel as per our discussion this evening...

Comment by Nigel Deakin [ 14/Jan/11 ]

These messages

[#|2011-01-13T11:14:06.600+0530|WARNING|glassfish3.1|javax.jms|_ThreadID=29;_ThreadName=Thread-1;|[I500]: Caught JVM Exception: java.io.EOFException: Trying to read 72 bytes. Already read 0 bytes.|#]

are warning messages. No errors or other messages were seen and there are no reports of the application behaving unexpectedly other than these messages.

Other users have reported receiving these messages periodically in earlier versions of MQ. There is a discussion here:
http://markmail.org/message/hh2ejjuxp5mo6njp#query:+page:1+mid:rumu3mg7unqbgm25+state:results
to which MQ team members responded.

This warning message is logged by the MQ client thread that is reading messages from a socket connected to the broker. (Note that even though the broker is embedded, since the broker is clustered, the client uses TCP to connect to the broker). The exception suggests that the client had received a "zero-length packet" from the broker. This message is often seen when the broker has failed, though this is not the case here. In that email discussion the MQ technical lead speculates that it is "probably a side effect of destination limits or TTL" and suggests that if there are no other problems the messages can be ignored.

Note that after this exception has been logged the client will attempt to recover the connection and carry on. This appears to have been the case here since no further messages were logged.

This is just a warning message, and is logged as such. Arguably GlassFish should never log warnings, but to suppress it might cause useful information to be lost if there are other problems. So it is proposed to close this bug as "not being a bug".

Comment by varunrupela [ 17/Jan/11 ]

Based on the analysis, the issue can be waived for GF 3.1. The issue should remain open as it appears in GF build 37 and as the MQ team will fix it in the long-term. It is useful to keep this issue open as opposed to creating a new one, since it contains quite some information around the bug.

Comment by Nigel Deakin [ 17/Jan/11 ]

Adding 3_1-exclude 3_1-release-notes tags. Note that this issue now only concerns the EOFExceptions and no other messages.

The release note should mention that very occasionally WARNING messages "java.io.EOFException: Trying to read 72 bytes. Already read 0 bytes" may be observed in the server log. If no other messages or exceptions are logged at the same time in either the server or broker logs these messages may be ignored.

Comment by varunrupela [ 18/Jan/11 ]

set the target release

Comment by easarina [ 18/Jan/11 ]

Was used b38 01/14. richAccess + SSL stress test on Win 2008 machines. Saw multiple "java.io.EOFException: Trying to read 72 bytes. Already read 0 bytes" warnings in server.log files. Was used jdk 1.6.0_23, 64 bit.

Comment by Nigel Deakin [ 19/Jan/11 ]

Updated summary to make it clear that it relates to warning messages, not exceptions.

Comment by Scott Fordin [ 23/Mar/11 ]

Need more info to add issue to 3.1 Release Notes. Is a release note still necessary?

Comment by Nigel Deakin [ 25/Mar/11 ]

Please add the following text (which I gave in my comment on 17 Jan) to the release note:

Issue 15424: Very occasionally WARNING messages "java.io.EOFException: Trying to read 72 bytes. Already read 0 bytes" may be observed in the server log. If no other messages or exceptions are logged at the same time in either the server or broker logs these messages may be ignored.

Nigel

Comment by Scott Fordin [ 15/Apr/11 ]

Added issue to 3.1 Release Notes.

Comment by Nigel Deakin [ 14/Dec/11 ]

Excluding from 3.1.2 for the same reason it was excluded from 3.1.1. It should continue to be mentioned in the release note.

Comment by Rebecca Parks [ 24/Jan/12 ]

If it's in the 3.1.1 Release Notes, it's carried over to 3.1.2 unless it's fixed in 3.1.2. There's no need to flag it for 3.1.2.

Comment by Nigel Deakin [ 15/Feb/13 ]

In accordance with the project triage guidelines this is not needed for 4.0 and so has been deferred until 4.0.1. Setting "fix version" accordingly.





[GLASSFISH-15675] Win2008, the machines had 2 interfaces & resolve a hostname to different IP addresses, MQ cluster is not configured to start correctly Created: 24/Jan/11  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: None
Fix Version/s: 4.1

Type: Bug Priority: Major
Reporter: easarina Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File domain.xml.asqe-oblade-15.das     File domain.xml.asqe-oblade-15.in3     File domain.xml.bigapp-oblade-1.in1     File domain.xml.bigapp-oblade-2.in2     Text File log.asqe-oblade-15     Text File log.bigapp-oblade-1     Text File log.bigapp-oblade-2     File log.txt.asqe-oblade-15.in3     File log.txt.bigapp-oblade-2.in2     File log.txt.bigapp-onlade-1.in1     File server.log.asaqe-oblade-15.das     File server.log.asqe-oblade-15.in3     File server.log.bigapp-oblade-2.in2     File server.log.bigapp-oblsde=1.in1    
Tags: 3_1_1-next, 3_1_1-scrubbed, 3_1_x-exclude, bj-reviewed

 Description   

Installed b39 01/24 on three Windows 2008 machines and created a cluster with three instances.
On machine asqe-oblade-15 was DAS + in3
On bigapp-oblade-1 was in1
On bigapp-oblade-2 was in2

All machines had two interfaces:
====================================================
asqe-oblade-15:

Ethernet adapter Local Area Connection 4:

Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::45ab:2b84:5a4b:5573%17
Autoconfiguration IPv4 Address. . : 169.254.85.115
Subnet Mask . . . . . . . . . . . : 255.255.0.0
Default Gateway . . . . . . . . . :

Ethernet adapter Local Area Connection 3:

Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::c534:cb5:b3d0:c360%15
IPv4 Address. . . . . . . . . . . : 10.133.184.219
Subnet Mask . . . . . . . . . . . : 255.255.248.0
Default Gateway . . . . . . . . . : 10.133.184.1
==============================================================
bigapp-oblade-1

Ethernet adapter Local Area Connection 4:

Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::7019:1cf8:a9d8:5bf%15
Autoconfiguration IPv4 Address. . : 169.254.5.191
Subnet Mask . . . . . . . . . . . : 255.255.0.0
Default Gateway . . . . . . . . . :

Ethernet adapter Local Area Connection 3:

Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::952f:b761:6e84:9806%13
IPv4 Address. . . . . . . . . . . : 10.133.184.150
Subnet Mask . . . . . . . . . . . : 255.255.248.0
Default Gateway . . . . . . . . . : 10.133.184.1
=======================================================

bigapp-oblade-2

Ethernet adapter Local Area Connection 4:

Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::9470:82fc:3090:bca1%17
Autoconfiguration IPv4 Address. . : 169.254.188.161
Subnet Mask . . . . . . . . . . . : 255.255.0.0
Default Gateway . . . . . . . . . :

Ethernet adapter Local Area Connection 3:

Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::5520:3687:8953:eadc%15
IPv4 Address. . . . . . . . . . . : 10.133.184.174
Subnet Mask . . . . . . . . . . . : 255.255.248.0
Default Gateway . . . . . . . . . : 10.133.184.1
============================================================

Because of the configuration with two interfaces, MQ brokers did not start correctly, can not connect with master broker and so on.

Please see MQ logs.

I believe we either need to find how to configure MQ, to be able to start brokers correctly on Win machines with two interfaces, or to document that we don't support such configuration.



 Comments   
Comment by amyk [ 26/Jan/11 ]

1 of the machines resolved a same hostname to different IP addresses from the other 2 machines. Either the machines need to setup to resolve a hostname to the same IP address or GlassFish/JMS should be configured to use explicit IP addresses (if supported)

Comment by Ed Bratt [ 26/Jan/11 ]

All machines should be configured to resolve hostnames the same way. Can you fix the machine setup and retest? If not, can you show how this has been tested successfully in the past?

Comment by easarina [ 26/Jan/11 ]

1) How I should configure the machines please specify?
2) How to configure JMS to use the explicit IP address?

Comment by Satish Kumar [ 27/Jan/11 ]

I tried this on a Windows Vista machine with both the LAN and Wireless adapters enabled but I was not able to reproduce this issue.

Comment by Nazrul [ 27/Jan/11 ]

Getting help from Amy.

Comment by Ed Bratt [ 27/Jan/11 ]

Direct IP setup is described here:
(Sorry, it's an internal link at the moment)
http://icon.red.iplanet.com/export7/mq4.5/demo/admingui/glassfish-cluster-mq-default-mode-explicit-ip/gf31-mq45-defaultmode-cluster-explicit-ip-demo.html

Comment by amyk [ 27/Jan/11 ]

Explicitly set network interface to be used on multi-homed computers is documented in MQ Administration Guide on configuring broker conventional cluster. If explicit network interface is not set, "connections between brokers might not succeed and as a result, the cluster will not be established"
http://download.oracle.com/docs/cd/E19798-01/821-1794/aeoia/index.html

Comment by easarina [ 28/Jan/11 ]

I've followed the provided instruction. Only I setup the correct IP addresses under nodes not node agents. WAs used Local JMS mode

Then I've restarted several times domain and cluster. But then saw the same MQ issue and, for example, richAccess was not deployed correctly. I've attached domain.xml files, server.log files, mq log files

Comment by amyk [ 28/Jan/11 ]

Reassign back to Satish to provide updated instruction to QE to configure GlassFish/JMS to use explicit IP addresses for a cluster running on multiple machines.

Comment by Nazrul [ 01/Feb/11 ]

We need to document the steps for customer to use GlassFish 3.1 (JMS in this case) on a multi-home machine. In 3.2, we may improve the out-of-the-box experience.

Comment by Harshad Vilekar [ 02/Feb/11 ]

The instructions are updated to use default (EMBEDDED) mode broker, rather than using LOCAL mode.

Use explicit IP address on multi-homed computer having multiple interface cards. GlassFish JMS module needs to set the property "imq.hostname" - and pass it on to the MQ broker in embedded / local mode. Currently, explicit IP address is not handled end to end. The workaround is to edit Message Queue configuration file, and assign the IP address to "imq.hostname" property for each clustered broker. Please document / release note this until this issue is fixed.

-------------------------
1. For Multi Node cluster with MQ EMBEDDED mode: Assign explicit IP address to "imq.hostname" property for each clustered broker. The setup steps are documented at:
http://icon.red.iplanet.com/export7/mq4.5/demo/admingui/glassfish-cluster-mq-default-mode-explicit-ip/gf31-mq45-defaultmode-cluster-explicit-ip-demo.html

2. For MQ in REMOTE mode:

  • Use explicit ip addresses when setting these broker properties: imq.hostname, imq.cluster.masterbroker, imq.cluster.brokerlist
  • Also, when using JMS Connection Factory, use explicit IP addresses for "AddressList" property

For details, see Configuring and Managing Broker Clusters: http://download.oracle.com/docs/cd/E19798-01/821-1794/6nmolieds/index.html

3. If using LOCAL mode, the brokers start up OK when configured to explicit IP (and imq.hostname is set). However, the broker shutdown is not handled properly - and this leaves the MQ brokers running when GlassFish cluster is shutdown. So, it is best not to use LOCAL mode when deploying GlassFish On a multi-homed computer.
-------------------------

Comment by Nazrul [ 02/Feb/11 ]

We expect a segment of users to use LOCAL mode. So, we need a fix for this issue.

Comment by Nazrul [ 04/Feb/11 ]

This will not make the RC2 cutoff. So, marking it to be release noted.

Based on conversation with Satish and Sudipa, when the Win 2008 setup is available, we may try imq commands to see if we can come up with manual steps for users. Please update the bug when ready so that the information can be included in the release note.

Comment by Scott Fordin [ 24/Mar/11 ]

Need more info to include in the Release Notes.

Comment by Nazrul [ 21/Apr/11 ]

It would be useful to take a look at this during 3.1.1.

Comment by scatari [ 29/Apr/11 ]

Approved.

Comment by Satish Kumar [ 27/Jun/11 ]

Not targeting this for 3.1.1 since it requires critical changes and needs adequate QE support to test this on various platforms. The changes required to support multi-homed setups can also cause regressions.

Comment by Satish Kumar [ 08/Jul/11 ]

Downgrading the priority of this issue to Major since this is not critical for the 3.1.1 release. We plan to fix this in the next release.

Comment by easarina [ 18/Jul/11 ]

I believe we need to document a work around.

Comment by Satish Kumar [ 08/Dec/11 ]

As stated before this issue requires critical changes and needs adequate QE support to test this on various platforms. The changes required to support multi-homed setups can also cause regressions. Hence, not targeting this for 3.1.2





[GLASSFISH-4151] Integrated OpenMQ/glassfish admin console Created: 11/Feb/08  Updated: 08/Mar/12

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: V3
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: rampsarathy Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 4,151
Status Whiteboard:

v3-prd-item


 Description   

This issue has been created to track the task of integrating the open mq admin
console with glassfish V3



 Comments   
Comment by rampsarathy [ 11/Feb/08 ]

Tagged as v3-prd-item

Comment by rampsarathy [ 29/May/08 ]

Assigning to Satish as he will be working on JMS features in V3

Comment by Tom Mueller [ 06/Mar/12 ]

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.

Comment by David Zhao [ 08/Mar/12 ]

Need more detail information about the enhancement.





[GLASSFISH-15934] Add support for multi-homed machines when using JMS Created: 10/Feb/11  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1_b41
Fix Version/s: 4.1

Type: Improvement Priority: Major
Reporter: Nigel Deakin Assignee: David Zhao
Resolution: Unresolved Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: bj-reviewed-t3

 Description   

GlassFish users need to be able to deploy GlassFish instances which use managed MQ brokers on multi-homed machines. The following use cases need to be supported:

1. Standalone and clustered (conventional using master broker, conventional using shared config db, and enhanced) brokers

2. EMBEDDED (for conventional and standalone) and LOCAL (for conventional, enhanced and standalone) brokers

3. Where the administrator wishes to configure the MQ broker to accept connections on ALL network interfaces for all broker services including the port mapper and the JMS service, and irrespective of whether port mapper connections are being handled by a Grizzly proxy or by the broker itself.

4. Where the administrator wishes to configure the MQ broker to SPECIFY the network interface used by all broker services including the port mapper and the JMS service, and irrespective of whether port mapper connections are being handled by a Grizzly proxy or by the broker itself.

It should be possible to perform all necessary configuration using asadmin or the GlassFish admin console, and perform remote deployment using "nodes" in the normal way.

In case (3) above, the comment made by user Foobartender in issue 11254 needs to be addressed:
http://java.net/jira/browse/GLASSFISH-11254?focusedCommentId=299711&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_299711



 Comments   
Comment by jthoennes [ 06/Jun/12 ]

Will this be corrected for GF 4.0?

Comment by bqin [ 19/Oct/12 ]

Defer it to 4.0.1 so we can focus on other issues targeted for 4.0.





[GLASSFISH-16051] XAStress Test fails with Glassfish / LOCAL MQ Created: 18/Feb/11  Updated: 14/Nov/11

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1_b43
Fix Version/s: None

Type: Bug Priority: Major
Reporter: mathim Assignee: Jagadish
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

glassfish mq


Attachments: Text File broker.log     Text File broker_stacktrace.log     Text File glassfish_stacktrace.log     Text File server.log     Text File server.log    
Tags: 3_1-exclude, 3_1-next, 3_1_1-scrubbed, 3_1_2-exclude

 Description   

XAStress test is an glassfish/mq integration stress test.

the test passes on glassfish/mq with MQ running as Embedded (default setup)

but running the test with glassfish and MQ as a LOCAL broker (non-default), the test failed after 72 hours (72 hours is the criteria for declaring the test pass)

But the problem can be consistently reproduced in a shorter timeframe by running the test on faster systems.

The problem
-----------

After running the XAStress for a day or two, glassfish server briefly becomes unresponsive to asadmin commands and hangs for about 10 - 15 minutes.

After this time, there are exceptions on the glassfish server.log and there is loss of a message in jms which causes the test to fail.

Listing the jms destination on the MQ broker shows the following.

-----------------------------------------------------------------------------------------------
Name Type State Producers Consumers Msgs
Total Wildcard Total Wildcard Count Remote UnAck Avg Size
-----------------------------------------------------------------------------------------------
MDB_XAT_TOPIC Topic RUNNING 0 0 1 0 1 0 1 260.0
mq.sys.dmq Queue RUNNING 0 - 0 - 0 0 0 0.0

Successfully listed destinations.



 Comments   
Comment by mathim [ 18/Feb/11 ]

Found the following exceptions on the glassfish server.

[#|2011-02-18T03:43:29.579-0800|WARNING|glassfish3.1|javax.resourceadapter.mqjmsra.inbound.message|_ThreadID=89;_ThreadName=Thread-1;|MQJMSRA_RA2001: Exception on stopMessageConsumer:Ignoring:
javax.resource.ResourceException: MQRA:EC:Error on closing MessageConsumer
at com.sun.messaging.jms.ra.EndpointConsumer.stopRemoteMessageConsumer(EndpointConsumer.java:661)
at com.sun.messaging.jms.ra.EndpointConsumer.stopMessageConsumer(EndpointConsumer.java:627)
at com.sun.messaging.jms.ra.ResourceAdapter.endpointDeactivation(ResourceAdapter.java:531)
at com.sun.enterprise.connectors.inbound.ConnectorMessageBeanClient.close(ConnectorMessageBeanClient.java:339)
at com.sun.ejb.containers.MessageBeanContainer$ASyncClientShutdownTask.run(MessageBeanContainer.java:912)
at com.sun.ejb.containers.MessageBeanContainer.cleanupResources(MessageBeanContainer.java:878)
at com.sun.ejb.containers.MessageBeanContainer.doConcreteContainerShutdown(MessageBeanContainer.java:948)
at com.sun.ejb.containers.BaseContainer.undeploy(BaseContainer.java:4200)
at org.glassfish.ejb.startup.EjbApplication.stop(EjbApplication.java:309)
at org.glassfish.internal.data.EngineRef.stop(EngineRef.java:169)
at org.glassfish.internal.data.ModuleInfo.stop(ModuleInfo.java:302)
at org.glassfish.internal.data.ApplicationInfo.stop(ApplicationInfo.java:314)
at com.sun.enterprise.v3.server.ApplicationLifecycle.unload(ApplicationLifecycle.java:997)
at com.sun.enterprise.v3.server.ApplicationLifecycle.undeploy(ApplicationLifecycle.java:1023)
at org.glassfish.deployment.admin.UndeployCommand.execute(UndeployCommand.java:333)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:355)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:370)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1067)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1200(CommandRunnerImpl.java:96)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1247)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1235)
at com.sun.enterprise.v3.admin.AdminAdapter.doCommand(AdminAdapter.java:465)
at com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:222)
at com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:168)
at com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:117)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:234)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:822)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:719)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1013)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.sun.messaging.jms.JMSException: MQRA:OMRP:Didnot finish waiting for OMRs to return:null
at com.sun.messaging.jms.ra.OnMessageRunnerPool.waitForAllOnMessageRunners(OnMessageRunnerPool.java:229)
at com.sun.messaging.jms.ra.MessageListener.waitForAllOnMessageRunners(MessageListener.java:168)
at com.sun.messaging.jms.ra.EndpointConsumer.stopRemoteMessageConsumer(EndpointConsumer.java:647)
... 39 more
Caused by: java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.sun.messaging.jms.ra.OnMessageRunnerPool.waitForAllOnMessageRunners(OnMessageRunnerPool.java:226)
... 41 more

#]

[#|2011-02-18T03:43:29.588-0800|SEVERE|glassfish3.1|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=86;_ThreadName=Thread-1;|javax.resource.ResourceException: MQRA:EC:Error on closing MessageConsumer
at com.sun.messaging.jms.ra.EndpointConsumer.stopRemoteMessageConsumer(EndpointConsumer.java:661)
at com.sun.messaging.jms.ra.EndpointConsumer.stopMessageConsumer(EndpointConsumer.java:627)
at com.sun.messaging.jms.ra.ResourceAdapter.endpointDeactivation(ResourceAdapter.java:531)
at com.sun.enterprise.connectors.inbound.ConnectorMessageBeanClient.close(ConnectorMessageBeanClient.java:339)
at com.sun.ejb.containers.MessageBeanContainer$ASyncClientShutdownTask.run(MessageBeanContainer.java:912)
at com.sun.ejb.containers.MessageBeanContainer.cleanupResources(MessageBeanContainer.java:878)
at com.sun.ejb.containers.MessageBeanContainer.doConcreteContainerShutdown(MessageBeanContainer.java:948)
at com.sun.ejb.containers.BaseContainer.undeploy(BaseContainer.java:4200)
at org.glassfish.ejb.startup.EjbApplication.stop(EjbApplication.java:309)
at org.glassfish.internal.data.EngineRef.stop(EngineRef.java:169)
at org.glassfish.internal.data.ModuleInfo.stop(ModuleInfo.java:302)
at org.glassfish.internal.data.ApplicationInfo.stop(ApplicationInfo.java:314)
at com.sun.enterprise.v3.server.ApplicationLifecycle.unload(ApplicationLifecycle.java:997)
at com.sun.enterprise.v3.server.ApplicationLifecycle.undeploy(ApplicationLifecycle.java:1023)
at org.glassfish.deployment.admin.UndeployCommand.execute(UndeployCommand.java:333)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:355)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:370)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1067)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1200(CommandRunnerImpl.java:96)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1247)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1235)
at com.sun.enterprise.v3.admin.AdminAdapter.doCommand(AdminAdapter.java:465)
at com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:222)
at com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:168)
at com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:117)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:234)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:822)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:719)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1013)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.sun.messaging.jms.JMSException: MQRA:OMRP:Didnot finish waiting for OMRs to return:null
at com.sun.messaging.jms.ra.OnMessageRunnerPool.waitForAllOnMessageRunners(OnMessageRunnerPool.java:229)
at com.sun.messaging.jms.ra.MessageListener.waitForAllOnMessageRunners(MessageListener.java:168)
at com.sun.messaging.jms.ra.EndpointConsumer.stopRemoteMessageConsumer(EndpointConsumer.java:647)
... 39 more
Caused by: java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.sun.messaging.jms.ra.OnMessageRunnerPool.waitForAllOnMessageRunners(OnMessageRunnerPool.java:226)
... 41 more

#]

[#|2011-02-18T03:43:29.589-0800|WARNING|glassfish3.1|javax.enterprise.system.container.ejb.mdb.com.sun.ejb.containers|_ThreadID=86;_ThreadName=Thread-1;|[MDBContainer] Current thread done cleanup()... |#]

[#|2011-02-18T03:43:29.594-0800|INFO|glassfish3.1|javax.enterprise.system.container.appclient.org.glassfish.appclient.server.core|_ThreadID=86;_ThreadName=Thread-1;|ACDEPL104: Java Web Start services stopped for the app client XATStress/app-client-ic.jar|#]

[#|2011-02-18T03:43:29.627-0800|SEVERE|glassfish3.1|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=89;_ThreadName=Thread-1;|service exception
java.lang.RuntimeException: ClientAbortException: java.nio.channels.ClosedChannelException
at com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:256)
at com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:168)
at com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:117)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:234)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:822)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:719)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1013)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
at java.lang.Thread.run(Thread.java:662)
Caused by: ClientAbortException: java.nio.channels.ClosedChannelException
at com.sun.grizzly.tcp.http11.GrizzlyOutputBuffer.doFlush(GrizzlyOutputBuffer.java:439)
at com.sun.grizzly.tcp.http11.GrizzlyOutputBuffer.flush(GrizzlyOutputBuffer.java:405)
at com.sun.grizzly.tcp.http11.GrizzlyOutputStream.flush(GrizzlyOutputStream.java:140)
at com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:253)
... 17 more
Caused by: java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
at com.sun.grizzly.util.OutputWriter.flushChannel(OutputWriter.java:108)
at com.sun.grizzly.util.OutputWriter.flushChannel(OutputWriter.java:76)
at com.sun.grizzly.http.SocketChannelOutputBuffer.flushChannel(SocketChannelOutputBuffer.java:326)
at com.sun.grizzly.http.SocketChannelOutputBuffer.flushBuffer(SocketChannelOutputBuffer.java:398)
at com.sun.grizzly.http.SocketChannelOutputBuffer.flush(SocketChannelOutputBuffer.java:376)
at com.sun.grizzly.http.ProcessorTask.action(ProcessorTask.java:1241)
at com.sun.grizzly.tcp.Response.action(Response.java:268)
at com.sun.grizzly.tcp.http11.GrizzlyOutputBuffer.doFlush(GrizzlyOutputBuffer.java:434)
... 20 more

#]
Comment by mathim [ 21/Feb/11 ]

attaching the log files and the stack trace for glassfish / mq broker

Comment by amyk [ 21/Feb/11 ]

The imqcmd output in the Description indicates the message had been delivered to client consumer and the broker was waiting for ack.

Comment by Chris Kasso [ 21/Feb/11 ]

Issue excluded, submitted after cutoff.

Comment by Nigel Deakin [ 22/Feb/11 ]

This test case involves an app client that writes messages to a topic, and two MDBs (1 CMT, 1 BMT) that consume these messages (unpon receiving a message, sending a reply to a reply queue and a reply topic respectively, then if on even message count or JMSRedelivered true , commit, else rollback the transaction). The app client then receives the reply messages. If the app client doesn't receive the expected messages within a timeout period the test script undeploys the application.

The above exception occurred when the application was undeployed. It shows that one or more MDB threads were blocked, thereby preventing application shutdown.

The thread dumps were logged after this point, and didn't show anything significant.

To find the root cause of this problem we need thread dumps before the application is undeployed. This might explain why the MDBs stopped processing messages and why the undeploy failed.

We are re-running these tests now to obtained the required information.

Comment by mathim [ 22/Feb/11 ]

attach server log with more lines

Comment by amyk [ 22/Feb/11 ]

The attached server log (the one with more lines) shows following exception on JMSRA delivering a message to MDB.

[#|2011-02-16T15:54:36.762-0800|INFO|glassfish3.1|javax.resourceadapter.mqjmsra.inbound.message|_ThreadID=85;_ThreadName=Thread-1;|MQJMSRA_MR1101: onMessage:WorkException-error code: 1 on omrId=29
javax.resource.spi.work.WorkRejectedException: error code: 1
at com.sun.enterprise.connectors.work.WorkCoordinator.workTimedOut(WorkCoordinator.java:278)
at com.sun.enterprise.connectors.work.WorkCoordinator.lock(WorkCoordinator.java:365)
at com.sun.enterprise.connectors.work.CommonWorkManager.startWork(CommonWorkManager.java:278)
at com.sun.enterprise.connectors.work.CommonWorkManager.startWork(CommonWorkManager.java:247)
at com.sun.enterprise.connectors.work.WorkManagerProxy.startWork(WorkManagerProxy.java:90)
at com.sun.messaging.jms.ra.OnMessageRunner.onMessage(OnMessageRunner.java:384)
at com.sun.messaging.jms.ra.MessageListener.onMessage(MessageListener.java:216)
at com.sun.messaging.jmq.jmsclient.MessageConsumerImpl.deliverAndAcknowledge(MessageConsumerImpl.java:358)
at com.sun.messaging.jmq.jmsclient.MessageConsumerImpl.onMessage(MessageConsumerImpl.java:287)
at com.sun.messaging.jmq.jmsclient.SessionReader.deliver(SessionReader.java:119)
at com.sun.messaging.jmq.jmsclient.ConsumerReader.run(ConsumerReader.java:192)
at java.lang.Thread.run(Thread.java:662)

#]
Comment by Nigel Deakin [ 03/Mar/11 ]

The javax.resource.spi.work.WorkRejectedException: error code: 1 is a timeout, which shouldn't happen in a call to Workmanager.startWork with no timeout.

Reassigning to Jagadish to investigate further.





[GLASSFISH-17051] Show JMS status Created: 15/Jul/11  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1.1_b11
Fix Version/s: 4.1

Type: New Feature Priority: Major
Reporter: mkarg Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Win7 Pro SP1 64 Bit de_DE



 Description   

Sometimes we would be glad if the admin gui would show the status of JMS objects. For example, how many connections are currently open and what client IP address opened it. Or, what length has a queue? Or whether a particular message is getting re-sent lots of times. Or, how long a queue is. Or whether the DMQ is empty or not and if not, what the content is.



 Comments   
Comment by Anissa Lam [ 19/Oct/12 ]

Assign to JMS team. When this is implemented and the statistics is available, please transfer to admingui to finish the UI work.

Comment by bqin [ 19/Oct/12 ]

Defer it with other related monitoring issues to 4.0.1.





[GLASSFISH-17049] glassfish-resources.xml: jms-connection-factory and jms-destination Created: 15/Jul/11  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1.1_b11
Fix Version/s: 4.1

Type: Improvement Priority: Major
Reporter: mkarg Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Win7 Pro SP1 64 Bit de_DE



 Description   

The browser based admin GUI allows the creation of JMS Connection Factory and JMS Destination Resoury with very few clicks and by provision of minimum values.

The glassfish-resource.xml file has no such comfort, and forces deployers to scan the manuals for the names of the jmsra properties for learn how to configure the same resources using the descriptor file.

It would be much more comfortable to have the following possibility:

<jms-connection-factory pool-name="MyPool" resource-type="javax.jms.ConnectionFactory" max-pool-size="1000" />

<jms-destination jndi-name="MyTopic" physical-destination-name="MyTopic" resource-type="javax.jms.Topic" />



 Comments   
Comment by Hong Zhang [ 15/Jul/11 ]

assign to jagadish for evaluation

Comment by Jagadish [ 15/Jul/11 ]

glassfish-resources.xml's configuration is based on <resources> configuration in domain.xml.

The configuration of jms-resource should be defined by jms team. Transferring to jms category.





[GLASSFISH-17298] Support weblogic *-jms.xml in GlassFish Created: 14/Sep/11  Updated: 13/Feb/13

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: future release

Type: Bug Priority: Major
Reporter: sonialiu Assignee: Nigel Deakin
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

windows


Attachments: File echoMDB.ear     Text File server.log    
Tags: 3_1_2-exclude

 Description   

report the bug for backward compatibility requirement for future BG product

The current version of GlassFish doesn't support weblogic *-jms.xml.
I tried to deploy the attached weblogic jms test application echoMDB.ear to GlassFish v3.1.1 and got the following error:
"Error occurred during deployment: Exception while loading the app : EJB Container initialization error. Please see server.log for more details."

In the server.log it displayed the following exceptions:
[#|2011-09-14T14:29:50.593-0700|SEVERE|glassfish3.1.1|javax.enterprise.resource.jms.com.sun.enterprise.connectors.jms.system|_ThreadID=17;_ThreadName=Thread-2;|Missing Destination JNDI Name|#]

[#|2011-09-14T14:29:50.593-0700|SEVERE|glassfish3.1.1|javax.enterprise.system.container.ejb.mdb.com.sun.ejb.containers|_ThreadID=17;_ThreadName=Thread-2;|MDB00017: [EchoMDB]: Exception in creating message-driven bean container: [com.sun.appserv.connectors.internal.api.ConnectorRuntimeException: Error in Runtime DD: missing destination JNDI name]|#]

[#|2011-09-14T14:29:50.593-0700|SEVERE|glassfish3.1.1|javax.enterprise.system.container.ejb.mdb.com.sun.ejb.containers|_ThreadID=17;_ThreadName=Thread-2;|com.sun.appserv.connectors.internal.api.ConnectorRuntimeException
com.sun.appserv.connectors.internal.api.ConnectorRuntimeException: Error in Runtime DD: missing destination JNDI name
at com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter.updateMDBRuntimeInfo(ActiveJmsResourceAdapter.java:1855)
at com.sun.enterprise.connectors.inbound.ConnectorMessageBeanClient.setup(ConnectorMessageBeanClient.java:186)
at com.sun.ejb.containers.MessageBeanContainer.<init>(MessageBeanContainer.java:205)
at com.sun.ejb.containers.ContainerFactoryImpl.createContainer(ContainerFactoryImpl.java:121)
at org.glassfish.ejb.startup.EjbApplication.loadContainers(EjbApplication.java:230)
at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:290)
at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:101)
at org.glassfish.internal.data.ModuleInfo.load(ModuleInfo.java:186)
at org.glassfish.internal.data.ApplicationInfo.load(ApplicationInfo.java:257)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:461)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:240)
at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:382)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:355)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:370)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1064)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1200(CommandRunnerImpl.java:96)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1244)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1232)
at org.glassfish.admin.rest.ResourceUtil.runCommand(ResourceUtil.java:202)
at org.glassfish.admin.rest.ResourceUtil.runCommand(ResourceUtil.java:195)
at org.glassfish.admin.rest.resources.TemplateListOfResource.createResource(TemplateListOfResource.java:148)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:134)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:134)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
at com.sun.jersey.server.impl.container.grizzly.GrizzlyContainer._service(GrizzlyContainer.java:182)
at com.sun.jersey.server.impl.container.grizzly.GrizzlyContainer.service(GrizzlyContainer.java:147)
at org.glassfish.admin.rest.adapter.RestAdapter.service(RestAdapter.java:184)
at com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:168)
at com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:117)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:238)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:828)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:725)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
at java.lang.Thread.run(Thread.java:619)

#]

[#|2011-09-14T14:29:50.593-0700|SEVERE|glassfish3.1.1|javax.enterprise.system.tools.deployment.org.glassfish.deployment.common|_ThreadID=17;_ThreadName=Thread-2;|Exception while invoking class org.glassfish.ejb.startup.EjbDeployer load method
java.lang.RuntimeException: EJB Container initialization error
at org.glassfish.ejb.startup.EjbApplication.loadContainers(EjbApplication.java:242)
at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:290)
at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:101)
at org.glassfish.internal.data.ModuleInfo.load(ModuleInfo.java:186)
at org.glassfish.internal.data.ApplicationInfo.load(ApplicationInfo.java:257)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:461)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:240)
at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:382)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:355)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:370)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1064)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1200(CommandRunnerImpl.java:96)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1244)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1232)
at org.glassfish.admin.rest.ResourceUtil.runCommand(ResourceUtil.java:202)
at org.glassfish.admin.rest.ResourceUtil.runCommand(ResourceUtil.java:195)
at org.glassfish.admin.rest.resources.TemplateListOfResource.createResource(TemplateListOfResource.java:148)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:134)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:134)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
at com.sun.jersey.server.impl.container.grizzly.GrizzlyContainer._service(GrizzlyContainer.java:182)
at com.sun.jersey.server.impl.container.grizzly.GrizzlyContainer.service(GrizzlyContainer.java:147)
at org.glassfish.admin.rest.adapter.RestAdapter.service(RestAdapter.java:184)
at com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:168)
at com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:117)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:238)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:828)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:725)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
at java.lang.Thread.run(Thread.java:619)
Caused by: com.sun.appserv.connectors.internal.api.ConnectorRuntimeException: Error in Runtime DD: missing destination JNDI name
at com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter.updateMDBRuntimeInfo(ActiveJmsResourceAdapter.java:1855)
at com.sun.enterprise.connectors.inbound.ConnectorMessageBeanClient.setup(ConnectorMessageBeanClient.java:186)
at com.sun.ejb.containers.MessageBeanContainer.<init>(MessageBeanContainer.java:205)
at com.sun.ejb.containers.ContainerFactoryImpl.createContainer(ContainerFactoryImpl.java:121)
at org.glassfish.ejb.startup.EjbApplication.loadContainers(EjbApplication.java:230)
... 55 more

#]

---------------------------------------------------------
I unjar-ed the echoMDB.ear and found that there is a descriptor file named echoMDB-jms.xml which weblogic server uses to define jndi-name, etc., for the application. Seems GlassFish ignored it during deployment.
Here is the snapshot of the echoMDB-jms.xml
<?xml version="1.0" encoding="UTF-8"?>

<weblogic-jms xmlns="http://www.bea.com/ns/weblogic/90">
<connection-factory name="ConnectionFactory"/>
<queue name="EchoQueue">
<sub-deployment-name>EchoQueue</sub-deployment-name>
<jndi-name>jms.echo.queue</jndi-name>
</queue>
<topic name="EchoTopic">
<sub-deployment-name>EchoQueue</sub-deployment-name>
</topic>
</weblogic-jms>
– attached server.log and echoMDB.ear



 Comments   
Comment by Hong Zhang [ 15/Sep/11 ]

Right, the weblogic-jms.xml is not supported today. Assign to Nigel for future planning for project BG.

Comment by sonialiu [ 21/Sep/11 ]

changed version to 4.0





[GLASSFISH-16467] Support for conventional peer broker clusters with Berkeley DB Created: 27/Apr/11  Updated: 23/Oct/12

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Satish Kumar Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: jms4candidate

 Description   

MQ 4.6 is introducing support for Oracle Berkeley DB as a persistent store type to address the elasticity requirements. BDB is a light-weight, highly scalable, embeddable, transactional database with a very low memory foot-print.

This requires changes to the configure-jms-cluster CLI command to enable users to configure this option and to the broker start-up properties that are passed in from the ActiveJMSResourceAdapter at start-up time.






[GLASSFISH-16466] Integrating MQ monitoring statistics with GF monitoring framework Created: 27/Apr/11  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: 4.1

Type: New Feature Priority: Major
Reporter: Satish Kumar Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The MQ broker provides a rich set of monitoring statistics that are currently not accessible through GF. A majority of these stats are implemented as JMX MBeans while a smaller number of them are only accessible through the MQ command line. In this release, we plan to provide access to these MQ monitoring statistics through the GF monitoring framework. The GF JMS module will implement the StatsProvider interface and will act as a proxy for these JMX MBeans. The primary focus will be on providing accessibility to the JMX enabled stats. Access to the non-JMX metrics will require code changes from MQ by either making these available through JMX or providing an alternative interface.



 Comments   
Comment by bqin [ 19/Oct/12 ]

This is not a must for 4.0 releaase,defer it with other monitoring issues to 4.0.1.





[GLASSFISH-16464] Message-store migration Created: 27/Apr/11  Updated: 10/Oct/12

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: Satish Kumar Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

In conventional MQ clusters, before a broker can be removed from the cluster, the message store of the broker needs to be migrated over to another broker to prevent loss of messages. This feature is required to support elasticity of MQ clusters.






[GLASSFISH-16811] JMS - connection refused on 7676 when domain uses 6767 Created: 06/Jun/11  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: None
Fix Version/s: 4.1

Type: Bug Priority: Major
Reporter: amyk Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to GLASSFISH-20606 create-domain subcommand doesn't writ... Open
Tags: 3_1-next, 3_1_1-scrubbed, 3_1_2-exclude

 Description   

The following is reported from admin@glassfish.java.net
http://java.net/projects/glassfish/lists/admin/archive/2011-06/message/9

--------------------------------
Hi everyone,

We have an application (J2EE) which is JMS based. In order to isolate it from other applications - we always set up a new domain with following port configuration:

ADMIN_PORT=4949

JMX_PORT=6868

JMS_PORT=6767

HTTP_PORT=8585

HTTPS_PORT=1818

ORB_PORT=7300

ORBS_PORT=8302

ORBM_PORT=9302

It turns out that sometimes I see JMS component trying to connect on JMS default port (7676) instead of configured one (6767):

[#|2011-05-23T11:48:29.917-0300|INFO|sun-appserver2.1|javax.resourceadapter.mqjmsra.lifecycle|_ThreadID=10;_ThreadName=main;|MQJMSRA_RA1101: SJSMQ JMS Resource Adapter starting...|#]
[#|2011-05-23T11:48:29.964-0300|INFO|sun-appserver2.1|javax.resourceadapter.mqjmsra.lifecycle|_ThreadID=10;_ThreadName=main;|MQJMSRA_LB1101: Looking for Broker Running at:localhost:6767|#]

[#|2011-05-23T11:49:44.529-0300|WARNING|sun-appserver2.1|javax.jms|_ThreadID=23;_ThreadName=p: thread-pool-1; w: 4;_RequestID=138cb0fe-ad3f-42c1-a11f-88d16c111043;|[C4003]: Error occurred on connection creation [localhost:7676]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-05-23T11:49:50.576-0300|WARNING|sun-appserver2.1|javax.jms|_ThreadID=23;_ThreadName=p: thread-pool-1; w: 4;_RequestID=138cb0fe-ad3f-42c1-a11f-88d16c111043;|[C4003]: Error occurred on connection creation [localhost:7676]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-05-23T11:49:56.482-0300|WARNING|sun-appserver2.1|javax.jms|_ThreadID=23;_ThreadName=p: thread-pool-1; w: 4;_RequestID=138cb0fe-ad3f-42c1-a11f-88d16c111043;|[C4003]: Error occurred on connection creation [localhost:7676]. - cause: java.net.ConnectException: Connection refused: connect|#]

Sometimes there are many messages like that in a row (Log file is attached).

I´m wondering if this something internal from Glassfish or something that application does - like a producer which is being initialized with wrong attibuttes.
Even within this error appearing our application communicate with other part.

Anyone already had similar situation?
Thanks in advance,

Márcio Geovani Jasinski
Blumenau, Santa Catarina
Fone: +55 47 9653 4899



 Comments   
Comment by amyk [ 14/Jun/11 ]

From Márcio Geovani Jasinski:

"Until now this seems to have no impact on JMS communication. We have another issue which is openmq shutdown. But I don't think both issues are related. Anyway, this is the discussion link just in case you would like to read and consider whether it is might be related:
http://www.java.net/forum/topic/glassfish/glassfish/openmq-unexpected-shutdown"

Comment by scatari [ 25/Jun/11 ]

Marking as to be considered for next release as according to the submitted report the functionality is not lost.

Comment by Satish Kumar [ 18/Nov/11 ]

It is difficult to diagnose what exactly is happening here without looking at the domain.xml. Please ensure that the jms-service entry is correct for server-config and should be as follows:
<jms-service default-jms-host="default_JMS_host" type="EMBEDDED">
<jms-host host="localhost" port="$

{JMS_PROVIDER_PORT}

" name="default_JMS_host"></jms-host>
</jms-service>

The only time it will try and connect to 7676 is if the jms-host.port is not specified. then it defaults to 7676.

Comment by David Zhao [ 06/Jun/13 ]

I cannot reproduce it with glassfish4 for the defect which was reported against a pretty old version. But I do see that create-domain subcommand doesn't deal with jms.port correctly, so default jms port 7676 could be used for a new domain with different jms.port property. Issue GLASSFISH-20606 was filed for that.





[GLASSFISH-20945] Glassfish 3.1.1 enhanced broker cluster Created: 08/Jan/14  Updated: 03/Jun/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: sp88 Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 4_0_1-reviewed

 Description   

Hi,

This is regarding glassfish IMQ JMS enhanced broker cluster.

The official documentation says that a enhanced broker cluster can be made by running the below command.

asadmin --passwordfile password-file configure-jms-cluster --clustertype=enhanced --configstoretype=shareddb --messagestoretype=jdbc --dbvendor database-vendor-name --dbuser database-user-name --dburl database-url
--property list-of-database-specific-properties glassfish-cluster-name

But it throws this error while running. "remote failure: configstoretype option is not configurable for Enhanced clusters."

Creating an enhanced broker cluster without starting glassfish at all is working fine. But then am doing that with imq commands. But i want my glassfish cluster to use this imq brokers.

The official documentation also says to start the imqbroker for the first time so that it creates the file glassfish/domains/domain1/imq/instances/imqbroker/props/config.properties. I have modified that config.properties file with the below stuff for db and broker id details.

imq.persist.jdbc.mysql.user=root
imq.cluster.ha=true
imq.persist.jdbc.dbVendor=mysql
imq.brokerid=broker1
imq.persist.jdbc.mysql.password=password
imq.persist.jdbc.vendorName.tableoption=CHARSET\=latin1
imq.persist.jdbc.mysql.property.url=jdbc\:mysql\://dbhost\:3306/mydb
imq.cluster.clusterid=mycluster
imq.persist.store=jdbc

But after starting glassfish and trying to do a imqcmd list bkr, am getting the following.

Error while connecting to the broker on host 'localhost' and port '7676'.
com.sun.messaging.jms.JMSException: [C4098]: Unknown broker service: admin
Please verify that there is a broker running on the specified host and port or
use the '-b' option to specify the correct broker host and port.

Please help me out in getting a glassfish IMQ enhanced broker cluster (where brokers are on different hosts) running. Please let me know if am doing something wrong here.

Regards
Sarath



 Comments   
Comment by David Zhao [ 09/Jan/14 ]

You are talking about 2 things here.

For the first one, it is like a doc defect to me about configure-jms-cluster. But the configstoretype and messagestoretype are ignored if clustertype is set to enhanced according to the command description (http://docs.oracle.com/cd/E26576_01/doc.312/e24938/configure-jms-cluster.htm#GSRFM00008). So you can just ignore those options when you are creating enhanced local broker cluster as a workaround.

For the second one, please make sure you are changing the correct imqbroker property files. In general, glassfish/domains/domain1/imq/instances/imqbroker/props/config.properties is for default server's use unless you add the server to the cluster specifically.

Normally, the steps to create a jms cluster of glassfish is as follows:

asadmin start-domain
asadmin create-cluster
asadmin configure-jms-cluster
asadmin create-local-instance --cluster ...
asadmin create-local-instance --cluster ...
asadmin start-cluster

Comment by sp88 [ 09/Jan/14 ]

HI David Zhao,

Thanks for your response.

So the thing am trying to achieve is this. I need a glassfish cluster with two instances (one local and one remote instance on another host). And this cluster must be enahanced broker cluster. Similar to instances, one broker must be on local instance and the other broker must be on the another host.

How am trying to do this. First am trying to create the config.properties file by starting imqbroker by using imqbrokerd command (this is official documented in glassfish)

Once the config.properties file is created, am modifying it with the following content so that a shareddb can be used by all brokers in the cluster (as soon as a new broker is added with the same DB config, it automatically gets detected)

imq.persist.jdbc.mysql.user=root
imq.cluster.ha=true
imq.persist.jdbc.dbVendor=mysql
imq.brokerid=broker1
imq.persist.jdbc.mysql.password=password
imq.persist.jdbc.vendorName.tableoption=CHARSET\=latin1
imq.persist.jdbc.mysql.property.url=jdbc\:mysql\://mydb.mydomain.com\:3306/vidispine
imq.cluster.clusterid=vsha
imq.persist.store=jdbc

But after adding the above stuff to the config file glassfish/domains/domain1/imq/instances/imqbroker/props/config.properties. Restarting glassfish domain is working fine. But am getting an error while listing the brokers locally.

Username: admin
Password:
Listing all the brokers in the cluster that the following broker is a member of:

-------------------------
Host Primary Port
-------------------------
localhost 7676

Error while connecting to the broker on host 'localhost' and port '7676'.
com.sun.messaging.jms.JMSException: [C4098]: Unknown broker service: admin
Please verify that there is a broker running on the specified host and port or
use the '-b' option to specify the correct broker host and port.

Listing the brokers in the cluster failed.

Can u help me get this done..Or am i doing some mistake here...

Thanks & Regards
sarath

Comment by David Zhao [ 10/Jan/14 ]

According to the error message, it seems no broker is running on port 7676. You can check the imq logs to see the broker status.

Comment by amyk [ 23/Feb/14 ]

If you start broker with imqbrokerd, then you need to configure the GlassFish JMS service to use REMOTE type, otherwise GlassFish instance will start its own broker instance on self-generated port if you didn't configure the port
http://docs.oracle.com/cd/E18930_01/html/821-2416/beaof.html#scrolltoc





[GLASSFISH-20714] GlassFish Broker's Use Of An RMI Registry Port Must Be Configurable And Documented Created: 19/Jul/13  Updated: 24/Apr/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: tecknobabble Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Generic


Issue Links:
Dependency
depends on GLASSFISH-20707 GlassFish V3 Embedded Broker fails to... Open
depends on GLASSFISH-20708 GlassFish V3 Broker fails to use the ... Resolved
depends on MQ-326 Embedded broker JMX support using Gla... Open

 Description   

This issue is related to GLASSFISH-20707 and GLASSFISH-20708.

The MQ broker's use of an RMI Registry port, in both embedded and local mode, needs to be something that can be controlled and configured by the end user, rather than having the resource adapter pick an arbitrary port of its own choosing - even if the value used can be predicted (mq broker port + 100).

The use of the RMI port also needs to be documented and explained and absolutely MUST be included in the list of known ports.

See: http://docs.oracle.com/cd/E26576_01/doc.312/e24939/release-notes.htm#ggpoq

which lists the MQ port mapper, admin and jms ports, and how the admin and jms ports can be configured. The same level of detail will be needed with the RMI Registry port.






[GLASSFISH-20707] GlassFish V3 Embedded Broker fails to use DAS GF JMX port (RmiRegistry) when its secured Created: 18/Jul/13  Updated: 04/Jun/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: gfuser9999 Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:
  • GFv3122
  • OS any
  • JDK any

Issue Links:
Dependency
depends on MQ-326 Embedded broker JMX support using Gla... Open
blocks GLASSFISH-20714 GlassFish Broker's Use Of An RMI Regi... Open
Tags: 4_0_1-reviewed, GF, MQ, RMIRegistry, extra, jms, jmx, port

 Description   

In GFv3, for the Admin-Connector (JMX connector) enable security-enable from false to true. When this is done, the the GlassFish MQ JMX will dynamically create a new RMIRegistry and unable to use GlassFish JMX port (8888)

Testcase:
---------
[Baseline]
1. Use a stock GF with embedded MQ.
2. Enable logger for javax.enterprise.resource.resourceadapter
& javax.enterprise.resource.jms.level to be FINE
3. Access the MQ by say "telneting to the MQ portmapper port 7676)
(since by default MQ is lazy initialized)
4. Now, you can see the MQ JMX port

  1. imq list jmx -b :7676
    should give you something like
    "service:jmx:rmi://<host>/jndi/rmi://<host>:8888/<host>/7676/jmxrmi"
    • Notice that the GF DAS RMI registry 8888 is used.

[JMX Admin-service security enabled] Now, secure the GF JMX connector

1. Restart, and redo step 3 & 4
2. # imq list jmx -b :7676 shows
"service:jmx:rmi://<host>/jndi/rmi://<host>:7776/<host>/7676/jmxrmi"

      • Note port 7776 (which is 7676+100 is created for the MQ rmi registry)

Concerns:
----------
a) More ports are used than needed.
b) The MQ port itself is +100 of the MQ portmapper (not changeable unless
using undocumented option).
Not settable too (always default to +100)

====
Logs:
====
[#|2013-07-17T12:17:55.167+0000|FINE|oracle-glassfish3.1.2|javax.enterprise.resource.jms.com.sun.enterprise.connectors.jms.system|_ThreadID=29;_ThreadName=Grizzly-kernel-thread(1);ClassName=com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter;MethodName=logFine;|isASRmiRegistryPortAvailable - JMSService Type:EMBEDDED|#]

[#|2013-07-17T12:17:55.170+0000|FINE|oracle-glassfish3.1.2|javax.enterprise.resource.jms.com.sun.enterprise.connectors.jms.system|_ThreadID=29;_ThreadName=Grizzly-kernel-thread(1);ClassName=com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter;MethodName=isASRmiRegistryPortAvailable;|Attempting to list rmi://0.0.0.0:8888|#]

[#|2013-07-17T12:17:55.181+0000|FINE|oracle-glassfish3.1.2|javax.enterprise.resource.jms.com.sun.enterprise.connectors.jms.system|_ThreadID=29;_ThreadName=Grizzly-kernel-thread(1);ClassName=com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter;MethodName=isASRmiRegistryPortAvailable;|non-JRMP server at remote endpoint rmi://0.0.0.0:8888|#]

      • Due to securing GFv3.x RMIRegistry, the testing of this fails
        with non-JRMP hence MQ create a new RMIRegistry for the MQ JMX.





invalid setting imq.cluster.masterbroker=mq: //localhost:27676/ for a broker running on different machine is passed to brokers in cluster (GLASSFISH-17278)

[GLASSFISH-20602] Don't user localhost as nodehost when creating node config for jms cluster (EMBEDDED ir LOCAL) Created: 04/Jun/13  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: None
Fix Version/s: 4.1

Type: Sub-task Priority: Major
Reporter: David Zhao Assignee: Mike Fitch
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

When configuring glassfish jms cluster with master broker in EMBEDDED or LOCAL mode, glassfish jms module will generate properties of 'imq.cluster.brokerlist' and 'imq.cluster.masterbroker' and passes those to MQ for bootstrap. For example,

imq.cluster.brokerlist=mq://host2:3076/,mq://host1:2076/
imq.cluster.masterbroker=mq://host1:2076/

The hostnames used in the properties are got from glassfish cluster nodes on which the glassfish instances reside. So at any time, please use unique hostnames which can be resolved at any physical servers on the network. Don't use localhost except that all the glassfish cluster instances are located in the same physical server.



 Comments   
Comment by dmatej [ 18/Jul/14 ]

At this moment there are no exceptions on GF 4.0.1-SNAPSHOT - imq broker refuses to start with localhost/127.0.0.1 in these properties:
ERROR [B3168]: Invalid broker address for this broker to run in cluster: Loopback IP address is not allowed in broker address mq://127.0.0.1:7679/?instName=ciswebclustercisweb1&brokerSessionUID=7687722948933731840&ha=false for cluster

But I got another problem - sometimes even the imq folder under the instance is missing. I don't know why ...

Comment by amyk [ 19/Jul/14 ]

The error B2168 is expected if the broker's imq.hostname is resolved to be the loopback address on the system. Please see MQ Adminsitration Guide on loopback address
http://docs.oracle.com/cd/E18930_01/html/821-2438/aeohz.html#aeoia

If no imq folder under instances, some possibilties: a) make sure that's the right location for the broker instance; b) The jms-service is lazy-init - run asadmin jms-ping or any imqcmd command will trigger the broker startup; c) The GlassFish server instance startup had a failure before init the jms-service or there was an error at very early stage of starting up the broker, please check the GlassFish server log

Comment by dmatej [ 21/Jul/14 ]

1) I don't understand why it is needed to block loopback IP - it only complicates testing with multiple instances in cluster on one machine. But okay, I have found workaround (changing hostname in node's host cfg and adding imq.hostname to JMS service properties, mapped to non-loopback IP).
2) The cause of my problems was in other thing, I created a issue for that: GLASSFISH-21138.
3) server.log contained bunch of nullpointers

Comment by amyk [ 25/Jul/14 ]

>why it is needed to block loopback IP
A broker on different system can join the cluster if that broker is configured such. Therefore loopback address is not allowed if a broker is configured to run in cluster environment.

> server.log contained bunch of nullpointers
Please file separate jira.

Comment by dmatej [ 28/Jul/14 ]

I understand why and how to configure broker to be accessible from different system.
But if loopback is not allowed, I cannot simply use broker from the cluster instances run on the same system (clusters created for testing). I don't think it is neccessary to forbid the loopback. It took me several days to bypass.

Comment by amyk [ 07/Sep/14 ]

A loopback IP address on one system can't be used as the 'address' of the system to other systems for communication.

You should be able to use a non loopback IP address and different ports to form a cluster on same system.

Comment by dmatej [ 07/Sep/14 ]

Once more again: then Glassfish fails to start, because IP address is changed every time I move to another network. Better approach is to let the configuration on the administrator - you can warn him about that, but you should allow that. I don't know about any other system which is so restrictive (and limited).

Comment by amyk [ 07/Sep/14 ]

Please file a jira to GlassFish server or GlassFish server JMS module to get real hostnames or real IP addresses of the GlassFish server instances when compose the GlassFish MQ cluster configuration properties

Comment by dmatej [ 07/Sep/14 ]

That issue already exists, but does not resolve the problem - there is no fixed hostname on notebook except the one resolved to loopback IP adress. All other names depend on a network DNS+DHCP. This is why I am writing it here and not to another issue or forum.

Comment by amyk [ 07/Sep/14 ]

It's the runtime hostname/IP addresses needed - not statically configured. The RE of GLASSFISH-17278 has determined it as a doc issue that how it currently works. You can reopen GLASSFISH-17278 or file a separate RFE jira so that it will go to the right responsible engineer. This jira sub-task is currently assigned to tech. writer.





[GLASSFISH-20742] One pager update: Add parameter --cascade to asadmin delete-jms-resource Created: 06/Aug/13  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.1
Fix Version/s: None

Type: Task Priority: Major
Reporter: David Zhao Assignee: Mike Fitch
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to GLASSFISH-19040 When asadmin delete-jms-resource is e... Resolved

 Description   

Every jms resource is stored as connector resource and connector connection pool. asadmin delete-jms-resource will delete underlying both connector resource and connector connection pool. If there are other connector resources sharing the same connector connection pool, the customer can use the new 'cascade' parameter to delete all associated connector resources. Please update the one pager of the command for --help.

--cascade
When set to true, all connector resources associated with the pool,
and the pool itself, are deleted. When set to false, the deletion
of pool fails if any resources are associated with the pool. The
resource must be deleted explicitly or the option must be set to
true. Default is false.






[GLASSFISH-20657] 'domain' is invalid value for --target parameter of command create-jms-resource if --restype is either javax.jms.Topic or javax.jms.Queue Created: 24/Jun/13  Updated: 24/Jun/13

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0_b89_RC5
Fix Version/s: None

Type: Task Priority: Major
Reporter: David Zhao Assignee: Mike Fitch
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Currently command create-jms-resource has the description in its one pager:

--target
Creates the JMS resource only for the specified target. Valid
values are as follows:

server
Creates the JMS resource for the default server instance. This
is the default value.

domain
Creates the JMS resource for the domain.

cluster-name
Creates the JMS resource for every server instance in the
specified cluster.

instance-name
Creates the JMS resource for the specified server instance.

Internally the command create-jms-resource invokes create-connector-resource for jms connection factory when --restype is javax.jms.ConnectionFactory, javax.jms.TopicConnectionFactory or javax.jms.QueueConnectionFactory.
Internally the command create-jms-resource invokes create-admin-object for jms destinations (Topic or Queue) when --restype is either javax.jms.Topic or javax.jms.Queue.

create-connector-resource accepts the same --target parameter values as create-jms-resource, which is fine.
But create-admin-object accepts the following --target parameter values, which doesn't contain 'domain' as create-jms-resource. So in the doc please mention that the command create-jms-resource doesn't accept 'domain' as the --target parameter value if --restype is either javax.jms.Topic or javax.jms.Queue.

--target
Specifies the target on which you are creating the administered
object. Valid values are as follows:

server
Creates the administered object for the default server instance
server and is the default value.

configuration_name
Creates the administered object for the named configuration.

cluster_name
Creates the administered object for every server instance in
the cluster.

instance_name
Creates the administered object for a particular server
instance.






[GLASSFISH-20650] NullPointerException in StringManagerBase, ActiveJmsResourceAdapter calls getString(...) with null as argument Created: 20/Jun/13  Updated: 20/Jun/13

Status: Open
Project: glassfish
Component/s: i18n, jms
Affects Version/s: 4.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: keil Assignee: michael.fang
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Java 1.7.0_21, 64bit
Windows 7, 64bit
GlassFish Server Open Source Edition 4.0 (build 89)



 Description   

ActiveJmsResourceAdapter calls StringManagerBase.getString("ajra.cannot_find_phy_dest", null) which raises a NullPointerException in the for-loop of StringManagerBase.getStringWithDefault(). See code snippets and stack trace below.

com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter
private String getPhysicalDestinationFromConfiguration(String logicalDest, String appName, String moduleName)
            throws ConnectorRuntimeException {
        
        ...

        if (ep == null) {
            String msg = sm.getString("ajra.cannot_find_phy_dest", null);
            throw new ConnectorRuntimeException(msg);
        }

        return ep.getValue();
    }
om.sun.enterprise.util.i18n.StringManagerBase
public String getStringWithDefault(String key, String defaultFormat, 
            Object arguments[]) {

        MessageFormat f =
            new MessageFormat( getStringWithDefault(key, defaultFormat) );

        for (int i=0; i<arguments.length; i++) {

            if ( arguments[i] == null ) {
java.lang.NullPointerException
	at com.sun.enterprise.util.i18n.StringManagerBase.getStringWithDefault(StringManagerBase.java:208)
	at com.sun.enterprise.util.i18n.StringManagerBase.getString(StringManagerBase.java:311)
	at com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter.getPhysicalDestinationFromConfiguration(ActiveJmsResourceAdapter.java:2163)
	at com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter.updateMDBRuntimeInfo(ActiveJmsResourceAdapter.java:1991)
	at com.sun.enterprise.connectors.inbound.ConnectorMessageBeanClient.setup(ConnectorMessageBeanClient.java:168)
	at org.glassfish.ejb.mdb.MessageBeanContainer.<init>(MessageBeanContainer.java:252)
	at org.glassfish.ejb.mdb.MessageBeanContainerFactory.createContainer(MessageBeanContainerFactory.java:63)
	at org.glassfish.ejb.startup.EjbApplication.loadContainers(EjbApplication.java:221)
	at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:291)
	at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:99)
	at org.glassfish.internal.data.ModuleInfo.load(ModuleInfo.java:206)
	at org.glassfish.internal.data.ApplicationInfo.load(ApplicationInfo.java:313)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:493)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:219)
	at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:491)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:527)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:523)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:356)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:522)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:546)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1423)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1500(CommandRunnerImpl.java:108)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1762)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1674)
	at com.sun.enterprise.v3.admin.AdminAdapter.doCommand(AdminAdapter.java:534)
	at com.sun.enterprise.v3.admin.AdminAdapter.onMissingResource(AdminAdapter.java:224)
	at org.glassfish.grizzly.http.server.StaticHttpHandler.service(StaticHttpHandler.java:297)
	at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:246)
	at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:191)
	at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:168)
	at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:189)
	at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
	at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:838)
	at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:722)





[GLASSFISH-21130] "Delete all messages" doesn't clear stats Created: 15/Jul/14  Updated: 16/Jul/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: vj_java Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

From the Glassfish server 4.0 admin console
server(Admin Server)>JMS Physical Destinations>Select any queue(s) -> "delete all messages"

after this If you click on "View" to see the statistics of the queue they remain unchanged.



 Comments   
Comment by David Zhao [ 15/Jul/14 ]

Are there any exceptions or error messages logged in server log?

Comment by vj_java [ 16/Jul/14 ]

No exceptions on server log





[GLASSFISH-21182] Thread deadlock in JMS broker Created: 03/Sep/14  Updated: 03/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: bdamer Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux



 Description   

We have run into a thread deadlock issue with one of our applications running on Glassfish 4 / MQ 5.0. It appears that the issue is caused by the message queue broker, so please let me know if this is not the right location to file this issue.

The scenario is as follows:

  • Broker is running locally with a single message queue set to FLOW_CONTROL
  • Application contains multiple producers, each within their own session, for said queue. These producers have different life-cycles, some being kept alive for the duration of the application, some being created and closed more frequently.
  • MDBs are used to consume messages from the queue

Under load, we see that after some time both the producer as well as the consumer side in our application start outputting messages that indicate that the broker is no longer responding:
[2014-09-03T03:46:15.339+0200] [glassfish 4.0] [WARNING] [] [javax.jms.connection] [tid: _ThreadID=411 _ThreadName=p: thread-pool-1; w: 111] [timeMillis: 1409708775339] [levelValue: 900] [[
[W2003]: Broker not responding [ACKNOWLEDGE(24)] for 1800 seconds. Still trying..., broker addr=localhost:7677(33136), connectionID=7124211059811362560, clientID=null, consumerID=14551566]]

It is however still possible to connect to the broker using imqcmd.

A thread dump of the broker shows the following:
Found one Java-level deadlock:
=============================
"Thread-jms[271]":
waiting to lock monitor 0x000000001366adc8 (object 0x00000000c01f94a8, a com.sun.messaging.jmq.jmsserver.core.Destination$ProducerFlow),
which is held by "Thread-jms[144]"
"Thread-jms[144]":
waiting to lock monitor 0x0000000013b81488 (object 0x00000000c0880b90, a com.sun.messaging.jmq.jmsserver.core.Producer),
which is held by "Thread-jms[271]"

Java stack information for the threads listed above:
===================================================
"Thread-jms[271]":
at com.sun.messaging.jmq.jmsserver.core.Destination$ProducerFlow.removeProducer(Destination.java:4951)

  • waiting to lock <0x00000000c01f94a8> (a com.sun.messaging.jmq.jmsserver.core.Destination$ProducerFlow)
    at com.sun.messaging.jmq.jmsserver.core.Destination.removeProducer(Destination.java:3670)
    at com.sun.messaging.jmq.jmsserver.core.Producer.destroyProducer(Producer.java:119)
  • locked <0x00000000c0880b90> (a com.sun.messaging.jmq.jmsserver.core.Producer)
    at com.sun.messaging.jmq.jmsserver.plugin.spi.ProducerSpi.destroyProducer(ProducerSpi.java:250)
    at com.sun.messaging.jmq.jmsserver.core.CoreLifecycleImpl.destroyProducer(CoreLifecycleImpl.java:347)
    at com.sun.messaging.jmq.jmsserver.service.imq.IMQConnection.removeProducer(IMQConnection.java:781)
    at com.sun.messaging.jmq.jmsserver.data.handlers.ProducerHandler.removeProducer(ProducerHandler.java:341)
    at com.sun.messaging.jmq.jmsserver.data.handlers.ProducerHandler.handle(ProducerHandler.java:214)
    at com.sun.messaging.jmq.jmsserver.data.PacketRouter.handleMessage(PacketRouter.java:199)
    at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.readData(IMQIPConnection.java:1320)
    at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.process(IMQIPConnection.java:553)
    at com.sun.messaging.jmq.jmsserver.service.imq.OperationRunnable.process(OperationRunnable.java:176)
    at com.sun.messaging.jmq.jmsserver.pool.BasicRunnable.run(BasicRunnable.java:499)
    at java.lang.Thread.run(Thread.java:744)
    "Thread-jms[144]":
    at com.sun.messaging.jmq.jmsserver.plugin.spi.ProducerSpi.isValid(ProducerSpi.java:261)
  • waiting to lock <0x00000000c0880b90> (a com.sun.messaging.jmq.jmsserver.core.Producer)
    at com.sun.messaging.jmq.jmsserver.core.Destination$ProducerFlow.checkResumeFlow(Destination.java:4879)
  • locked <0x00000000c01f94a8> (a com.sun.messaging.jmq.jmsserver.core.Destination$ProducerFlow)
  • locked <0x00000000c01f8db0> (a com.sun.messaging.jmq.util.lists.SimpleNFLHashMap)
    at com.sun.messaging.jmq.jmsserver.core.Destination$ProducerFlow.checkResumeFlow(Destination.java:4790)
    at com.sun.messaging.jmq.jmsserver.core.Destination.removeProducer(Destination.java:3671)
    at com.sun.messaging.jmq.jmsserver.core.Producer.destroyProducer(Producer.java:119)
  • locked <0x00000000c08a02d8> (a com.sun.messaging.jmq.jmsserver.core.Producer)
    at com.sun.messaging.jmq.jmsserver.plugin.spi.ProducerSpi.destroyProducer(ProducerSpi.java:250)
    at com.sun.messaging.jmq.jmsserver.core.CoreLifecycleImpl.destroyProducer(CoreLifecycleImpl.java:347)
    at com.sun.messaging.jmq.jmsserver.service.imq.IMQConnection.removeProducer(IMQConnection.java:781)
    at com.sun.messaging.jmq.jmsserver.data.handlers.ProducerHandler.removeProducer(ProducerHandler.java:341)
    at com.sun.messaging.jmq.jmsserver.data.handlers.ProducerHandler.handle(ProducerHandler.java:214)
    at com.sun.messaging.jmq.jmsserver.data.PacketRouter.handleMessage(PacketRouter.java:199)
    at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.readData(IMQIPConnection.java:1320)
    at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.process(IMQIPConnection.java:553)
    at com.sun.messaging.jmq.jmsserver.service.imq.OperationRunnable.process(OperationRunnable.java:176)
    at com.sun.messaging.jmq.jmsserver.pool.BasicRunnable.run(BasicRunnable.java:499)
    at java.lang.Thread.run(Thread.java:744)

Found 1 deadlock.

Looking at the source for OpenMQ 5.0, there seems to be a race condition to acquire a lock on a producer. As best as I understand, the scenario is as follows:

Request to destroy Producer A locks A
Request to destroy Producer B locks B
Destination locks flow control to destroy A
Destination waits for lock on flow control to destroy B
Destination for A attempts to acquire lock for producer B (synchronized call to Producer.isValid)






[GLASSFISH-21034] Glassfish randomly swaps queues Created: 08/Apr/14  Updated: 03/Jun/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0_b89_RC5
Fix Version/s: None

Type: Bug Priority: Major
Reporter: enschedem Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

MacOS


Tags: 4_0_1-reviewed

 Description   

I the following example I using 2 JMS queues in Glassfish 4.0. The first class sends a message to queue 1, this is picked up by a second bean which forwards the message to a next queue. From there it is picked up to the last bean. Schematically:

Creator -> jms/q1 -> QueueBean1 -> jms/q2 -> QueueBean2.

Problem is that QueueBean1 and QueueBean2 are receiving messages randomly from q1 and q2. It seems to me that q1 and q2 are just one queue and the MessageBeans a looking to that queue.



 Comments   
Comment by David Zhao [ 09/Apr/14 ]

How did you create jms/q1 and jms/q2? Please check if they are pointing to the same physical destination in MQ. If true, then the 2 MDBs are listening to the same Queue actually.





[GLASSFISH-20999] description attribute on Topic doesn't get set Created: 01/Mar/14  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: 4.1

Type: New Feature Priority: Major
Reporter: pranahata Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

When setting the description attribute of a Topic via @JMSDestinationDefinition or glassfish-resources, it doesn't show either on the client or on the server when doing a toString of the Topic object.



 Comments   
Comment by amyk [ 16/Apr/14 ]

Hi David, please evaluate this to see where the 'description' should be displayed for a JMSDestinationDefinition annotation as a GlassFish JMS resource. Neither Java EE specification nor JMS specification specifies this 'description' be a description for the associated JMS provider's Destination object, further JMS API javadoc only says following for Queue.toString() and Topic.toString(), hence this is not a bug in Topic.toString() from spec conformance point of view

/** Returns a string representation of this object.
 *
 * @return the provider-specific identity values for this topic
 */
Comment by David Zhao [ 16/Apr/14 ]

The JMS admin objects created via admin console or asadmin can take and show the description property. But the property is not included in Queue.toString() either.

Currently the JMS resources defined by annotations like @JMSDestinationDefinition are not shown in the admin gui although it is global scoped. So now the description property will not be seen anywhere in the admin gui, except it is in the codes to be as comments.

To summarize it,

1) It is same as normal JMS resources which are created traditionally that the description is not shown in the toString().
2) If it is decided that the resources defined by annotations are shown in admin gui in the funture, then the description will appear there.

Question to Amy,

Should the description go to Queue.imqDestinationDescription?

Comment by amyk [ 16/Apr/14 ]

>Should the description go to Queue.imqDestinationDescription?

Yes.

Comment by David Zhao [ 17/Apr/14 ]

Update:

create-jms-resource maps property Description to imqDestinationDescription, so it is included in dest.toString():

Oracle GlassFish(tm) Server MQ Destination
getName(): myQueue
Class: com.sun.messaging.Queue
getVERSION(): 3.0
isReadonly(): false
getProperties():

{imqDestinationName=myQueue, imqDestinationDescription=abcde}

create-admin-object is a JCA general command, so it doesn't do the property mapping for JMS. Admin console is using this command as REST endpoint too, so if the JMS destination is created in this way, the description will not be shown in dest.toString(). But the Description property might be stored in MQ destination's properties map, because it can be persisted/restored.

@JMSDestinationDefinition doesn't map Description <-> imqDestinationDescription, so it is not included in toString().





[GLASSFISH-20983] Deadlock when creating a JMS connection when Glassfish is starting Created: 13/Feb/14  Updated: 03/Jun/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: electricsam Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux, jdk 1.7.0_45


Tags: 4_0_1-reviewed

 Description   

Note: This is not reproducible in a standalone environment using the embedded broker.

In my environment I have a cluster with 2 nodes, using enhanced local brokers. I have also tried enhanced remote brokers with the same effect.

When glassfish is starting, but not completed, if an application tries to send a message deadlock occurs when creating the JMS connection. This can easily be reproduced by trying to send a message in a @Singleton @Startup bean.

Producer:

@ApplicationScoped
public class PerformanceLoggingProducer {

    private static final Logger logger = Logger.getLogger(PerformanceLoggingProducer.class.getName());

//    @Resource(mappedName = "jms/ConnectionFactory")
    private ConnectionFactory connectionFactory;
//    @Resource(name = "jms/PerformanceLoggingQueue")
    private Queue queue;

    public PerformanceLoggingProducer() {
        try {
            connectionFactory = (ConnectionFactory) new InitialContext().lookup("jms/ConnectionFactory");
            queue = (Queue) new InitialContext().lookup("jms/PerformanceLoggingQueue");
        } catch (NamingException ex) {
            logger.log(Level.SEVERE, null, ex);
        }
    }

    public void log(final PerformanceLogTracker tracker) {

       if (tracker != null) {

        try (Connection connection = connectionFactory.createConnection()) {
			Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            MessageProducer producer = session.createProducer(queue);
            ObjectMessage message = session.createObjectMessage(tracker);
            producer.send(message);
         } catch (Exception ex) {
                logger.log(Level.SEVERE, "An error occurred when logging message: " + tracker, ex);
         }
  

       }
    }

}

Thread dump:

Notice this:

"main" prio=10 tid=0x00007f5d5c00d000 nid=0xb04 in Object.wait() [0x00007f5d63f61000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000755814468> (a com.sun.jts.CosTransactions.EventSemaphore)
	at java.lang.Object.wait(Object.java:503)
	at com.sun.jts.CosTransactions.EventSemaphore.waitEvent(EventSemaphore.java:133)
	- locked <0x0000000755814468> (a com.sun.jts.CosTransactions.EventSemaphore)
	at com.sun.jts.CosTransactions.RecoveryManager.waitForResync(RecoveryManager.java:1455)
	at com.sun.jts.CosTransactions.TransactionFactoryImpl.localCreate(TransactionFactoryImpl.java:180)
2014-02-12 11:04:05
Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.45-b08 mixed mode):

"__ejb-thread-pool16" daemon prio=10 tid=0x00007f5d08302000 nid=0xc18 waiting on condition [0x00007f5d045ae000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool15" daemon prio=10 tid=0x00007f5d08300000 nid=0xc17 waiting on condition [0x00007f5d046af000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool14" daemon prio=10 tid=0x00007f5d082fe000 nid=0xc16 waiting on condition [0x00007f5d047b0000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool13" daemon prio=10 tid=0x00007f5d08063000 nid=0xc15 waiting on condition [0x00007f5d048b1000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool12" daemon prio=10 tid=0x00007f5d08061000 nid=0xc14 waiting on condition [0x00007f5d049b2000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool11" daemon prio=10 tid=0x00007f5d0805e800 nid=0xc13 waiting on condition [0x00007f5d04ab3000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool10" daemon prio=10 tid=0x00007f5d0805c800 nid=0xc12 waiting on condition [0x00007f5d04bb4000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool9" daemon prio=10 tid=0x00007f5d08752800 nid=0xc11 waiting on condition [0x00007f5d04cb5000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool8" daemon prio=10 tid=0x00007f5d08447000 nid=0xc10 waiting on condition [0x00007f5d04db6000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool7" daemon prio=10 tid=0x00007f5d08446000 nid=0xc0f waiting on condition [0x00007f5d04eb7000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool6" daemon prio=10 tid=0x00007f5d08496000 nid=0xc0e waiting on condition [0x00007f5d04fb8000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool5" daemon prio=10 tid=0x00007f5d0865a800 nid=0xc0d waiting on condition [0x00007f5d052bb000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool4" daemon prio=10 tid=0x00007f5d08659000 nid=0xc0c waiting on condition [0x00007f5d050b9000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool3" daemon prio=10 tid=0x00007f5d08657800 nid=0xc0b waiting on condition [0x00007f5d383c4000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool2" daemon prio=10 tid=0x00007f5d083d5800 nid=0xc0a waiting on condition [0x00007f5d385c6000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"__ejb-thread-pool1" daemon prio=10 tid=0x00007f5d083d4800 nid=0xc09 waiting on condition [0x00007f5d15e6a000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"Attach Listener" daemon prio=10 tid=0x00007f5d10022000 nid=0xc03 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"GMS-isConnected-Group-myproject-thread-3" daemon prio=10 tid=0x00007f5d541ee000 nid=0xbbc waiting on condition [0x00007f5d15c68000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000767174f80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-isConnected-Group-myproject-thread-2" daemon prio=10 tid=0x00007f5d541ed000 nid=0xbbb waiting on condition [0x00007f5d384c5000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000767174f80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-isConnected-Group-myproject-thread-1" daemon prio=10 tid=0x00007f5d541ec000 nid=0xbba waiting on condition [0x00007f5d386c7000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000767174f80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"Thread-52" daemon prio=10 tid=0x00007f5d3001d800 nid=0xbad runnable [0x00007f5d051ba000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
	- locked <0x000000075b302158> (a java.io.BufferedInputStream)
	at com.sun.jndi.ldap.Connection.run(Connection.java:853)
	at java.lang.Thread.run(Thread.java:744)

"JTS Resync Thread" daemon prio=10 tid=0x00007f5d5e5d8800 nid=0xbab in Object.wait() [0x00007f5d053bc000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000755814448> (a com.sun.jts.CosTransactions.EventSemaphore)
	at java.lang.Object.wait(Object.java:503)
	at com.sun.jts.CosTransactions.EventSemaphore.waitEvent(EventSemaphore.java:133)
	- locked <0x0000000755814448> (a com.sun.jts.CosTransactions.EventSemaphore)
	at com.sun.jts.CosTransactions.RecoveryManager.proceedWithXARecovery(RecoveryManager.java:948)
	at com.sun.jts.CosTransactions.RecoveryManager.recover(RecoveryManager.java:492)
	at com.sun.jts.CosTransactions.ResyncThread.run(RecoveryManager.java:1881)

"Recovery Helper Thread" daemon prio=10 tid=0x00007f5d5e5d7800 nid=0xbaa in Object.wait() [0x00007f5d054bd000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000075184dd10> (a org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:134)
	- locked <0x000000075184dd10> (a org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext)
	at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2204)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:579)
	at org.jvnet.hk2.internal.IterableProviderImpl.get(IterableProviderImpl.java:87)
	at com.sun.enterprise.resource.recovery.ConnectorsRecoveryResourceHandler.loadXAResourcesAndItsConnections(ConnectorsRecoveryResourceHandler.java:220)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.getAllRecoverableResources(ResourceRecoveryManagerImpl.java:212)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:240)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:337)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.postConstruct(ResourceRecoveryManagerImpl.java:108)
	at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:281)
	at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:328)
	at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:448)
	at org.jvnet.hk2.internal.SingletonContext.findOrCreate(SingletonContext.java:118)
	at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2204)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:579)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:566)
	at com.sun.jts.jta.TransactionServiceProperties$RecoveryHelperThread.run(TransactionServiceProperties.java:361)

"iMQReadChannel-3" daemon prio=10 tid=0x00007f5d5e74b800 nid=0xba9 runnable [0x00007f5d055be000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
	- locked <0x000000075b302b18> (a java.io.BufferedInputStream)
	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1836)
	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1804)
	at com.sun.messaging.jmq.io.Packet.readPacket(Packet.java:1731)
	- locked <0x000000075b302b40> (a com.sun.messaging.jmq.io.ReadWritePacket)
	at com.sun.messaging.jmq.io.ReadOnlyPacket.readPacket(ReadOnlyPacket.java:115)
	at com.sun.messaging.jmq.io.ReadWritePacket.readPacket(ReadWritePacket.java:70)
	- locked <0x000000075b302b40> (a com.sun.messaging.jmq.io.ReadWritePacket)
	at com.sun.messaging.jmq.jmsclient.protocol.SocketConnectionHandler.readPacket(SocketConnectionHandler.java:72)
	at com.sun.messaging.jmq.jmsclient.ProtocolHandler.readPacket(ProtocolHandler.java:1830)
	at com.sun.messaging.jmq.jmsclient.ReadChannel.run(ReadChannel.java:1275)
	at java.lang.Thread.run(Thread.java:744)

"imqConnectionFlowControl-3" daemon prio=10 tid=0x00007f5d5e74a000 nid=0xba7 in Object.wait() [0x00007f5d056bf000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000075b302de0> (a com.sun.messaging.jmq.jmsclient.FlowControl)
	at com.sun.messaging.jmq.jmsclient.FlowControl.run(FlowControl.java:307)
	- locked <0x000000075b302de0> (a com.sun.messaging.jmq.jmsclient.FlowControl)
	at java.lang.Thread.run(Thread.java:744)

"ContainerBackgroundProcessor[StandardEngine[glassfish-web].StandardHost[server].StandardContext[/ejb-timer-service-app]]" daemon prio=10 tid=0x00007f5d5e495800 nid=0xba5 waiting on condition [0x00007f5d05882000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1801)
	at java.lang.Thread.run(Thread.java:744)

"imqConsumerReader-2-563587832198206464-1" daemon prio=10 tid=0x00007f5d5e7be800 nid=0xba4 in Object.wait() [0x00007f5d05983000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000075d9000e0> (a com.sun.messaging.jmq.jmsclient.SessionQueue)
	at com.sun.messaging.jmq.jmsclient.SessionQueue.dequeueWait(SessionQueue.java:257)
	- locked <0x000000075d9000e0> (a com.sun.messaging.jmq.jmsclient.SessionQueue)
	at com.sun.messaging.jmq.jmsclient.ConsumerReader.run(ConsumerReader.java:168)
	at java.lang.Thread.run(Thread.java:744)

"iMQReadChannel-2" daemon prio=10 tid=0x00007f5d5e1c8800 nid=0xba3 runnable [0x00007f5d15f6b000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
	- locked <0x000000075d900a50> (a java.io.BufferedInputStream)
	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1836)
	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1804)
	at com.sun.messaging.jmq.io.Packet.readPacket(Packet.java:1731)
	- locked <0x000000075d900a78> (a com.sun.messaging.jmq.io.ReadWritePacket)
	at com.sun.messaging.jmq.io.ReadOnlyPacket.readPacket(ReadOnlyPacket.java:115)
	at com.sun.messaging.jmq.io.ReadWritePacket.readPacket(ReadWritePacket.java:70)
	- locked <0x000000075d900a78> (a com.sun.messaging.jmq.io.ReadWritePacket)
	at com.sun.messaging.jmq.jmsclient.protocol.SocketConnectionHandler.readPacket(SocketConnectionHandler.java:72)
	at com.sun.messaging.jmq.jmsclient.ProtocolHandler.readPacket(ProtocolHandler.java:1830)
	at com.sun.messaging.jmq.jmsclient.ReadChannel.run(ReadChannel.java:1275)
	at java.lang.Thread.run(Thread.java:744)

"imqConnectionFlowControl-2" daemon prio=10 tid=0x00007f5d5e1c7800 nid=0xba2 in Object.wait() [0x00007f5d15d69000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000075d900d18> (a com.sun.messaging.jmq.jmsclient.FlowControl)
	at com.sun.messaging.jmq.jmsclient.FlowControl.run(FlowControl.java:307)
	- locked <0x000000075d900d18> (a com.sun.messaging.jmq.jmsclient.FlowControl)
	at java.lang.Thread.run(Thread.java:744)

"ContainerBackgroundProcessor[StandardEngine[glassfish-web].StandardHost[server].StandardContext[/smb-access-svc]]" daemon prio=10 tid=0x00007f5d5e16c800 nid=0xb9b waiting on condition [0x00007f5d05ad9000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1801)
	at java.lang.Thread.run(Thread.java:744)

"imqConsumerReader-1-563587832187488512-0" daemon prio=10 tid=0x00007f5d5d645000 nid=0xb9a in Object.wait() [0x00007f5d05bda000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007675442c0> (a com.sun.messaging.jmq.jmsclient.SessionQueue)
	at com.sun.messaging.jmq.jmsclient.SessionQueue.dequeueWait(SessionQueue.java:257)
	- locked <0x00000007675442c0> (a com.sun.messaging.jmq.jmsclient.SessionQueue)
	at com.sun.messaging.jmq.jmsclient.ConsumerReader.run(ConsumerReader.java:168)
	at java.lang.Thread.run(Thread.java:744)

"iMQReadChannel-1" daemon prio=10 tid=0x00007f5d5db24000 nid=0xb99 runnable [0x00007f5d05cdb000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
	- locked <0x0000000767544bd8> (a java.io.BufferedInputStream)
	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1836)
	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1804)
	at com.sun.messaging.jmq.io.Packet.readPacket(Packet.java:1731)
	- locked <0x0000000767544c58> (a com.sun.messaging.jmq.io.ReadWritePacket)
	at com.sun.messaging.jmq.io.ReadOnlyPacket.readPacket(ReadOnlyPacket.java:115)
	at com.sun.messaging.jmq.io.ReadWritePacket.readPacket(ReadWritePacket.java:70)
	- locked <0x0000000767544c58> (a com.sun.messaging.jmq.io.ReadWritePacket)
	at com.sun.messaging.jmq.jmsclient.protocol.SocketConnectionHandler.readPacket(SocketConnectionHandler.java:72)
	at com.sun.messaging.jmq.jmsclient.ProtocolHandler.readPacket(ProtocolHandler.java:1830)
	at com.sun.messaging.jmq.jmsclient.ReadChannel.run(ReadChannel.java:1275)
	at java.lang.Thread.run(Thread.java:744)

"imqConnectionFlowControl-1" daemon prio=10 tid=0x00007f5d5db1e800 nid=0xb98 in Object.wait() [0x00007f5d05ddc000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000767544ef8> (a com.sun.messaging.jmq.jmsclient.FlowControl)
	at com.sun.messaging.jmq.jmsclient.FlowControl.run(FlowControl.java:307)
	- locked <0x0000000767544ef8> (a com.sun.messaging.jmq.jmsclient.FlowControl)
	at java.lang.Thread.run(Thread.java:744)

"GMS-DistributedStateCache-Group-myproject-thread-1" daemon prio=10 tid=0x00007f5d0d867800 nid=0xb97 waiting on condition [0x00007f5d05edd000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f68b90> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-validateMasterChangeEvents-Group-myproject-thread-1" daemon prio=10 tid=0x00007f5d1c01d000 nid=0xb96 waiting on condition [0x00007f5d05fde000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f68c40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-processNotify-Group-myproject-thread-2" daemon prio=10 tid=0x000000000078a000 nid=0xb95 waiting on condition [0x00007f5d060df000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65c98> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-processNotify-Group-myproject-thread-1" daemon prio=10 tid=0x0000000000789000 nid=0xb94 waiting on condition [0x00007f5d071f0000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65c98> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-McastMsgProcessor-Group-myproject-thread-10" daemon prio=10 tid=0x00007f5d2c042000 nid=0xb91 waiting on condition [0x00007f5d061e0000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"Timer-3" daemon prio=10 tid=0x00007f5d48016000 nid=0xb90 in Object.wait() [0x00007f5d062e1000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007675452e8> (a java.util.TaskQueue)
	at java.util.TimerThread.mainLoop(Timer.java:552)
	- locked <0x00000007675452e8> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:505)

"GMS-McastMsgProcessor-Group-myproject-thread-9" daemon prio=10 tid=0x00007f5d2c040000 nid=0xb8f waiting on condition [0x00007f5d063e2000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-McastMsgProcessor-Group-myproject-thread-8" daemon prio=10 tid=0x00007f5d2c03d800 nid=0xb8e waiting on condition [0x00007f5d064e3000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"Thread-27" prio=10 tid=0x00007f5d5d89e800 nid=0xb8d in Object.wait() [0x00007f5d065e4000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000767545468> (a com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive)
	at java.lang.Object.wait(Object.java:503)
	at com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive.run(Util.java:818)
	- locked <0x0000000767545468> (a com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive)

"GMS-McastMsgProcessor-Group-myproject-thread-7" daemon prio=10 tid=0x00007f5d2c03b800 nid=0xb8c waiting on condition [0x00007f5d066e5000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"SelectorThread" daemon prio=10 tid=0x00007f5d5da26000 nid=0xb8b runnable [0x00007f5d067e6000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
	- locked <0x0000000767545690> (a sun.nio.ch.Util$2)
	- locked <0x00000007675456a0> (a java.util.Collections$UnmodifiableSet)
	- locked <0x0000000767545648> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
	at com.sun.corba.ee.impl.transport.SelectorImpl.run(SelectorImpl.java:303)

"GMS-McastMsgProcessor-Group-myproject-thread-6" daemon prio=10 tid=0x00007f5d2c039800 nid=0xb8a waiting on condition [0x00007f5d068e7000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-McastMsgProcessor-Group-myproject-thread-5" daemon prio=10 tid=0x00007f5d2c038000 nid=0xb89 waiting on condition [0x00007f5d069e8000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-McastMsgProcessor-Group-myproject-thread-4" daemon prio=10 tid=0x00007f5d2c02b800 nid=0xb88 waiting on condition [0x00007f5d06ae9000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-McastMsgProcessor-Group-myproject-thread-3" daemon prio=10 tid=0x00007f5d2c029800 nid=0xb87 waiting on condition [0x00007f5d06bea000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-McastMsgProcessor-Group-myproject-thread-2" daemon prio=10 tid=0x00007f5d2c027800 nid=0xb86 waiting on condition [0x00007f5d06ceb000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS-McastMsgProcessor-Group-myproject-thread-1" daemon prio=10 tid=0x00007f5d2c04b000 nid=0xb85 waiting on condition [0x00007f5d06dec000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"GMS FailureVerifier Thread for Group-myproject" daemon prio=10 tid=0x00007f5d5cd84800 nid=0xb84 in Object.wait() [0x00007f5d06eed000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f68e88> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at com.sun.enterprise.mgmt.HealthMonitor$FailureVerifier.run(HealthMonitor.java:1430)
	- locked <0x0000000752f68e88> (a java.lang.Object)
	at java.lang.Thread.run(Thread.java:744)

"GMS InDoubtPeerDetector Thread for Group-myproject" daemon prio=10 tid=0x00007f5d5cd82000 nid=0xb83 in Object.wait() [0x00007f5d06fee000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f68f18> (a java.lang.Object)
	at com.sun.enterprise.mgmt.HealthMonitor$InDoubtPeerDetector.run(HealthMonitor.java:1290)
	- locked <0x0000000752f68f18> (a java.lang.Object)
	at java.lang.Thread.run(Thread.java:744)

"GMS HealthMonitor for Group-myproject" daemon prio=10 tid=0x00007f5d5cd80000 nid=0xb82 in Object.wait() [0x00007f5d070ef000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f68fa8> (a java.lang.Object)
	at com.sun.enterprise.mgmt.HealthMonitor.run(HealthMonitor.java:817)
	- locked <0x0000000752f68fa8> (a java.lang.Object)
	at java.lang.Thread.run(Thread.java:744)

"GMS MasterNode processOutstandingChangeEvents Group-myproject" daemon prio=10 tid=0x00007f5d5d247800 nid=0xb80 in Object.wait() [0x00007f5d072f1000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f690b0> (a java.util.TreeSet)
	at java.lang.Object.wait(Object.java:503)
	at com.sun.enterprise.mgmt.MasterNode$ProcessOutstandingMessagesTask.run(MasterNode.java:1859)
	- locked <0x0000000752f690b0> (a java.util.TreeSet)
	at java.lang.Thread.run(Thread.java:744)

"Timer-2" daemon prio=10 tid=0x00007f5d5d23e800 nid=0xb7f in Object.wait() [0x00007f5d073f2000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f69140> (a java.util.TaskQueue)
	at java.lang.Object.wait(Object.java:503)
	at java.util.TimerThread.mainLoop(Timer.java:526)
	- locked <0x0000000752f69140> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:505)

"IP Multicast Listener for /228.9.219.87:9336" daemon prio=10 tid=0x00007f5d5d279000 nid=0xb7e runnable [0x00007f5d074f3000]
   java.lang.Thread.State: RUNNABLE
	at java.net.PlainDatagramSocketImpl.receive0(Native Method)
	- locked <0x0000000752f691d8> (a java.net.PlainDatagramSocketImpl)
	at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145)
	- locked <0x0000000752f691d8> (a java.net.PlainDatagramSocketImpl)
	at java.net.DatagramSocket.receive(DatagramSocket.java:786)
	- locked <0x00000007694c5448> (a java.net.DatagramPacket)
	- locked <0x0000000752f69208> (a java.net.MulticastSocket)
	at com.sun.enterprise.mgmt.transport.BlockingIOMulticastSender.run(BlockingIOMulticastSender.java:254)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(20)" daemon prio=10 tid=0x00007f5d5d259800 nid=0xb7d in Object.wait() [0x00007f5d075f4000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(19)" daemon prio=10 tid=0x00007f5d5d257800 nid=0xb7c in Object.wait() [0x00007f5d076f5000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(18)" daemon prio=10 tid=0x00007f5d5d255800 nid=0xb7b in Object.wait() [0x00007f5d077f6000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(17)" daemon prio=10 tid=0x00007f5d5d2be800 nid=0xb7a in Object.wait() [0x00007f5d078f7000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(16)" daemon prio=10 tid=0x00007f5d5d2bc800 nid=0xb79 in Object.wait() [0x00007f5d079f8000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(15)" daemon prio=10 tid=0x00007f5d5d2ba800 nid=0xb78 in Object.wait() [0x00007f5d07af9000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(14)" daemon prio=10 tid=0x00007f5d5d2b8800 nid=0xb77 in Object.wait() [0x00007f5d07bfa000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(13)" daemon prio=10 tid=0x00007f5d5d2b6800 nid=0xb76 in Object.wait() [0x00007f5d07cfb000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(12)" daemon prio=10 tid=0x00007f5d5d2b4800 nid=0xb75 in Object.wait() [0x00007f5d07dfc000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(11)" daemon prio=10 tid=0x00007f5d5d2b2000 nid=0xb74 in Object.wait() [0x00007f5d07efd000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(10)" daemon prio=10 tid=0x00007f5d5d2b0000 nid=0xb73 in Object.wait() [0x00007f5d07ffe000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(9)" daemon prio=10 tid=0x00007f5d5d2ae000 nid=0xb72 in Object.wait() [0x00007f5d141d8000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(8)" daemon prio=10 tid=0x00007f5d5d2ac000 nid=0xb71 in Object.wait() [0x00007f5d142d9000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(7)" daemon prio=10 tid=0x00007f5d5d2aa000 nid=0xb70 in Object.wait() [0x00007f5d143da000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(6)" daemon prio=10 tid=0x00007f5d5d2a7800 nid=0xb6f in Object.wait() [0x00007f5d144db000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(5)" daemon prio=10 tid=0x00007f5d5d2a5800 nid=0xb6e in Object.wait() [0x00007f5d145dc000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(4)" daemon prio=10 tid=0x00007f5d5d2a3800 nid=0xb6d in Object.wait() [0x00007f5d146dd000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(3)" daemon prio=10 tid=0x00007f5d5d2a1800 nid=0xb6c in Object.wait() [0x00007f5d147de000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(2)" daemon prio=10 tid=0x00007f5d5d29f800 nid=0xb6b in Object.wait() [0x00007f5d148df000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS-GrizzlyControllerThreadPool-Group-myproject(1)" daemon prio=10 tid=0x00007f5d5d75a000 nid=0xb6a in Object.wait() [0x00007f5d149e0000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752f65f90> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000752f65f90> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"Grizzly-kernel(2) SelectorRunner" daemon prio=10 tid=0x00007f5d5d758800 nid=0xb69 runnable [0x00007f5d14ae1000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
	- locked <0x0000000752f66618> (a sun.nio.ch.Util$2)
	- locked <0x0000000752f66628> (a java.util.Collections$UnmodifiableSet)
	- locked <0x0000000752f665d0> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
	at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112)
	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:333)
	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:274)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"Grizzly-kernel(1) SelectorRunner" daemon prio=10 tid=0x00007f5d5d757000 nid=0xb68 runnable [0x00007f5d14be2000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
	- locked <0x0000000752f667e0> (a sun.nio.ch.Util$2)
	- locked <0x0000000752f667f0> (a java.util.Collections$UnmodifiableSet)
	- locked <0x0000000752f66798> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
	at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112)
	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:333)
	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:274)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"GMS ViewWindowThread Group-myproject" daemon prio=10 tid=0x00007f5d5d744800 nid=0xb67 waiting on condition [0x00007f5d14ce3000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f668e8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at com.sun.enterprise.ee.cms.impl.base.ViewWindowImpl.run(ViewWindowImpl.java:189)
	at java.lang.Thread.run(Thread.java:744)

"GMS MessageWindowThread Group-myproject" daemon prio=10 tid=0x00007f5d5d747000 nid=0xb66 waiting on condition [0x00007f5d14de4000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f669d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at com.sun.enterprise.ee.cms.impl.base.MessageWindow.run(MessageWindow.java:108)
	at java.lang.Thread.run(Thread.java:744)

"GMS SignalHandler for Group-myproject thread" daemon prio=10 tid=0x00007f5d5d2e1000 nid=0xb65 waiting on condition [0x00007f5d14ee5000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000752f69a40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at com.sun.enterprise.ee.cms.impl.common.SignalHandler.run(SignalHandler.java:90)
	at java.lang.Thread.run(Thread.java:744)

"process reaper" daemon prio=10 tid=0x00007f5d1000a000 nid=0xb62 runnable [0x00007f5d14f1e000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.UNIXProcess.waitForProcessExit(Native Method)
	at java.lang.UNIXProcess.access$200(UNIXProcess.java:54)
	at java.lang.UNIXProcess$3.run(UNIXProcess.java:174)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"process reaper" daemon prio=10 tid=0x00007f5d10008800 nid=0xb5e runnable [0x00007f5d14f57000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.UNIXProcess.waitForProcessExit(Native Method)
	at java.lang.UNIXProcess.access$200(UNIXProcess.java:54)
	at java.lang.UNIXProcess$3.run(UNIXProcess.java:174)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"process reaper" daemon prio=10 tid=0x00007f5d10007000 nid=0xb5a runnable [0x00007f5d3806f000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.UNIXProcess.waitForProcessExit(Native Method)
	at java.lang.UNIXProcess.access$200(UNIXProcess.java:54)
	at java.lang.UNIXProcess$3.run(UNIXProcess.java:174)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"process reaper" daemon prio=10 tid=0x00007f5d10005800 nid=0xb56 runnable [0x00007f5d380a8000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.UNIXProcess.waitForProcessExit(Native Method)
	at java.lang.UNIXProcess.access$200(UNIXProcess.java:54)
	at java.lang.UNIXProcess$3.run(UNIXProcess.java:174)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"connector-timer-proxy" daemon prio=10 tid=0x00007f5d5d80f800 nid=0xb54 in Object.wait() [0x00007f5d15058000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007563003f8> (a java.util.TaskQueue)
	at java.util.TimerThread.mainLoop(Timer.java:552)
	- locked <0x00000007563003f8> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:505)

"Timer-1" daemon prio=10 tid=0x00007f5d5d814000 nid=0xb53 in Object.wait() [0x00007f5d15159000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007563004c8> (a java.util.TaskQueue)
	at java.lang.Object.wait(Object.java:503)
	at java.util.TimerThread.mainLoop(Timer.java:526)
	- locked <0x00000007563004c8> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:505)

"p: thread-pool-1; w: 5" daemon prio=10 tid=0x00007f5d5d4ea000 nid=0xb52 in Object.wait() [0x00007f5d15460000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
	at com.sun.corba.ee.impl.threadpool.WorkQueueImpl.requestWork(WorkQueueImpl.java:124)
	- locked <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:518)

"p: thread-pool-1; w: 4" daemon prio=10 tid=0x00007f5d5d4e8000 nid=0xb51 in Object.wait() [0x00007f5d15561000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
	at com.sun.corba.ee.impl.threadpool.WorkQueueImpl.requestWork(WorkQueueImpl.java:124)
	- locked <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:518)

"p: thread-pool-1; w: 3" daemon prio=10 tid=0x00007f5d5d4e6000 nid=0xb50 in Object.wait() [0x00007f5d15662000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
	at com.sun.corba.ee.impl.threadpool.WorkQueueImpl.requestWork(WorkQueueImpl.java:124)
	- locked <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:518)

"p: thread-pool-1; w: 2" daemon prio=10 tid=0x00007f5d5d4e3800 nid=0xb4f runnable [0x00007f5d15763000]
   java.lang.Thread.State: RUNNABLE
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
	at java.net.ServerSocket.implAccept(ServerSocket.java:530)
	at sun.security.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:317)
	at com.sun.corba.ee.impl.transport.AcceptorImpl.getAcceptedSocket(AcceptorImpl.java:148)
	at com.sun.corba.ee.impl.transport.ListenerThreadImpl.doWork(ListenerThreadImpl.java:114)
	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497)
	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540)

"p: thread-pool-1; w: 1" daemon prio=10 tid=0x00007f5d5d4ed000 nid=0xb4e runnable [0x00007f5d15864000]
   java.lang.Thread.State: RUNNABLE
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
	at java.net.ServerSocket.implAccept(ServerSocket.java:530)
	at sun.security.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:317)
	at com.sun.corba.ee.impl.transport.AcceptorImpl.getAcceptedSocket(AcceptorImpl.java:148)
	at com.sun.corba.ee.impl.transport.ListenerThreadImpl.doWork(ListenerThreadImpl.java:114)
	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497)
	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540)

"dol-jar-scanner" daemon prio=10 tid=0x00007f5d5d0d0800 nid=0xb4d waiting on condition [0x00007f5d15965000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x000000075276f448> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"dol-jar-scanner" daemon prio=10 tid=0x00007f5d5d0c7800 nid=0xb4c waiting on condition [0x00007f5d15a66000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x000000075276f448> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"Timer-0" daemon prio=10 tid=0x00007f5d5d28a800 nid=0xb4b in Object.wait() [0x00007f5d15b67000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000755f002e8> (a java.util.TaskQueue)
	at java.util.TimerThread.mainLoop(Timer.java:552)
	- locked <0x0000000755f002e8> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:505)

"pool-11-thread-1" daemon prio=10 tid=0x00007f5d5d376000 nid=0xb46 waiting on condition [0x00007f5d1606c000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007522dfeb8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:162)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"pool-10-thread-1" daemon prio=10 tid=0x00007f5d5d374000 nid=0xb45 waiting on condition [0x00007f5d1616d000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007518ea718> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:162)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"pool-9-thread-1" daemon prio=10 tid=0x00007f5d5cd01000 nid=0xb44 waiting on condition [0x00007f5d381c2000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007522df948> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:162)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"pool-8-thread-1" daemon prio=10 tid=0x00007f5d5c877000 nid=0xb43 waiting on condition [0x00007f5d382c3000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007522dfb48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:162)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"ContainerBackgroundProcessor[StandardEngine[glassfish-web].StandardHost[server].StandardContext[/highcharts-export-web]]" daemon prio=10 tid=0x00007f5d5d1fd800 nid=0xb3d waiting on condition [0x00007f5d388df000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1801)
	at java.lang.Thread.run(Thread.java:744)

"pool-2-thread-1" prio=10 tid=0x00007f5d5d1fc000 nid=0xb3c waiting on condition [0x00007f5d389e0000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000753b00068> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"ContainerBackgroundProcessor[StandardEngine[glassfish-web]]" daemon prio=10 tid=0x00007f5d5cee6800 nid=0xb3b waiting on condition [0x00007f5d38ae1000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1801)
	at java.lang.Thread.run(Thread.java:744)

"ContainerBackgroundProcessor[StandardEngine[glassfish-web].StandardHost[server].StandardContext[]]" daemon prio=10 tid=0x00007f5d5ceb4800 nid=0xb3a waiting on condition [0x00007f5d38be2000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1801)
	at java.lang.Thread.run(Thread.java:744)

"deployment-jar-scanner" daemon prio=10 tid=0x00007f5d5c937800 nid=0xb39 waiting on condition [0x00007f5d39487000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007558ce390> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"deployment-jar-scanner" daemon prio=10 tid=0x00007f5d5c92e000 nid=0xb38 waiting on condition [0x00007f5d39588000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007558ce390> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"transaction-manager" daemon prio=10 tid=0x00007f5d5c899000 nid=0xb37 in Object.wait() [0x00007f5d39689000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000755500308> (a java.util.TaskQueue)
	at java.lang.Object.wait(Object.java:503)
	at java.util.TimerThread.mainLoop(Timer.java:526)
	- locked <0x0000000755500308> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:505)

"admin-listener-kernel(1) SelectorRunner" daemon prio=10 tid=0x00007f5d5c7c9800 nid=0xb36 runnable [0x00007f5d3978a000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
	- locked <0x00000007555012d0> (a sun.nio.ch.Util$2)
	- locked <0x00000007555012e0> (a java.util.Collections$UnmodifiableSet)
	- locked <0x0000000755501288> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
	at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112)
	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:333)
	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:274)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"admin-listener-0" daemon prio=10 tid=0x00007f5d5c7c8000 nid=0xb35 in Object.wait() [0x00007f5d3988b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555003a0> (a java.lang.Object)
	at org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:172)
	- locked <0x00000007555003a0> (a java.lang.Object)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"admin-listener(5)" daemon prio=10 tid=0x00007f5d5c7c5800 nid=0xb34 in Object.wait() [0x00007f5d3998c000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555004b8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x00000007555004b8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"admin-listener(4)" daemon prio=10 tid=0x00007f5d5c7c3800 nid=0xb33 in Object.wait() [0x00007f5d39a8d000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555004b8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x00000007555004b8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"admin-listener(3)" daemon prio=10 tid=0x00007f5d5c7c1800 nid=0xb32 in Object.wait() [0x00007f5d39b8e000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555004b8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x00000007555004b8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"admin-listener(2)" daemon prio=10 tid=0x00007f5d5c7bc800 nid=0xb31 in Object.wait() [0x00007f5d39c8f000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555004b8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x00000007555004b8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"admin-listener(1)" daemon prio=10 tid=0x00007f5d5c7ba800 nid=0xb30 in Object.wait() [0x00007f5d39d90000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555004b8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x00000007555004b8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-2-kernel(1) SelectorRunner" daemon prio=10 tid=0x00007f5d5c79e800 nid=0xb2f runnable [0x00007f5d39e91000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
	- locked <0x0000000755501600> (a sun.nio.ch.Util$2)
	- locked <0x0000000755501610> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000007555015b8> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
	at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112)
	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:333)
	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:274)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-2-0" daemon prio=10 tid=0x00007f5d5c79d000 nid=0xb2e in Object.wait() [0x00007f5d39f92000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555006c0> (a java.lang.Object)
	at org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:172)
	- locked <0x00000007555006c0> (a java.lang.Object)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-2(5)" daemon prio=10 tid=0x00007f5d5c79a000 nid=0xb2d in Object.wait() [0x00007f5d3a093000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555007d8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x00000007555007d8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-2(4)" daemon prio=10 tid=0x00007f5d5c798000 nid=0xb2c in Object.wait() [0x00007f5d3a194000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555007d8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x00000007555007d8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-2(3)" daemon prio=10 tid=0x00007f5d5c796000 nid=0xb2b in Object.wait() [0x00007f5d3a295000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555007d8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x00000007555007d8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-2(2)" daemon prio=10 tid=0x00007f5d5c794000 nid=0xb2a in Object.wait() [0x00007f5d5815b000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555007d8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x00000007555007d8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-2(1)" daemon prio=10 tid=0x00007f5d5c782000 nid=0xb29 in Object.wait() [0x00007f5d5825c000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555007d8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x00000007555007d8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-1-kernel(1) SelectorRunner" daemon prio=10 tid=0x00007f5d5c75e800 nid=0xb28 runnable [0x00007f5d5835d000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
	- locked <0x0000000755501918> (a sun.nio.ch.Util$2)
	- locked <0x0000000755501928> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000007555018d0> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
	at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112)
	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:333)
	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:274)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-1-0" daemon prio=10 tid=0x00007f5d5c757800 nid=0xb27 in Object.wait() [0x00007f5d5845e000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007555009e0> (a java.lang.Object)
	at org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:172)
	- locked <0x00000007555009e0> (a java.lang.Object)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-1(5)" daemon prio=10 tid=0x00007f5d5c754800 nid=0xb26 in Object.wait() [0x00007f5d5855f000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000755500af8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000755500af8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-1(4)" daemon prio=10 tid=0x00007f5d5c752800 nid=0xb25 in Object.wait() [0x00007f5d58660000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000755500af8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000755500af8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-1(3)" daemon prio=10 tid=0x00007f5d5c750800 nid=0xb24 in Object.wait() [0x00007f5d58761000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000755500af8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000755500af8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-1(2)" daemon prio=10 tid=0x00007f5d5c74e800 nid=0xb23 in Object.wait() [0x00007f5d58862000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000755500af8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000755500af8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"http-listener-1(1)" daemon prio=10 tid=0x00007f5d5c73a800 nid=0xb22 in Object.wait() [0x00007f5d58963000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000755500af8> (a java.lang.Object)
	at java.lang.Object.wait(Object.java:503)
	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
	- locked <0x0000000755500af8> (a java.lang.Object)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
	at java.lang.Thread.run(Thread.java:744)

"Grizzly-HttpSession-Expirer" daemon prio=10 tid=0x00007f5d5c707000 nid=0xb21 waiting on condition [0x00007f5d58c6c000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000755500c40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"Thread-4" daemon prio=10 tid=0x00007f5d5c5d0000 nid=0xb20 waiting on condition [0x00007f5d58d6d000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x000000075184d880> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at com.sun.common.util.logging.LoggingOutputStream.log(LoggingOutputStream.java:146)
	at com.sun.common.util.logging.LoggingOutputStream$1.run(LoggingOutputStream.java:125)

"Thread-3" daemon prio=10 tid=0x00007f5d5c5bd800 nid=0xb1f waiting on condition [0x00007f5d58e6e000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x000000075184d990> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at com.sun.common.util.logging.LoggingOutputStream.log(LoggingOutputStream.java:146)
	at com.sun.common.util.logging.LoggingOutputStream$1.run(LoggingOutputStream.java:125)

"Thread-2" daemon prio=10 tid=0x00007f5d5c5ca000 nid=0xb1e waiting on condition [0x00007f5d58f6f000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x000000075184daa0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at com.sun.enterprise.server.logging.GFFileHandler.log(GFFileHandler.java:825)
	at com.sun.enterprise.server.logging.GFFileHandler$3.run(GFFileHandler.java:540)

"pool-1-thread-1" daemon prio=10 tid=0x00007f5d5c49e800 nid=0xb1d waiting on condition [0x00007f5d591c4000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x000000075184dc00> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)

"FelixStartLevel" daemon prio=10 tid=0x00007f5d5c2f2800 nid=0xb1c in Object.wait() [0x00007f5d59e66000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000075094af88> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:503)
	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:279)
	- locked <0x000000075094af88> (a java.util.ArrayList)
	at java.lang.Thread.run(Thread.java:744)

"FelixDispatchQueue" daemon prio=10 tid=0x00007f5d08152800 nid=0xb18 in Object.wait() [0x00007f5d59d65000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752880f40> (a java.util.ArrayList)
	at java.lang.Object.wait(Object.java:503)
	at org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:1063)
	- locked <0x0000000752880f40> (a java.util.ArrayList)
	at org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:54)
	at org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:101)
	at java.lang.Thread.run(Thread.java:744)

"Service Thread" daemon prio=10 tid=0x00007f5d5c195000 nid=0xb15 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" daemon prio=10 tid=0x00007f5d5c192800 nid=0xb14 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" daemon prio=10 tid=0x00007f5d5c190000 nid=0xb13 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x00007f5d5c114800 nid=0xb12 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f5d5c113000 nid=0xb11 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x00007f5d5c0f2000 nid=0xb10 in Object.wait() [0x00007f5d5a5e4000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007528814e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
	- locked <0x00000007528814e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)

"Reference Handler" daemon prio=10 tid=0x00007f5d5c0ee000 nid=0xb0f in Object.wait() [0x00007f5d5a6e5000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000752881580> (a java.lang.ref.Reference$Lock)
	at java.lang.Object.wait(Object.java:503)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
	- locked <0x0000000752881580> (a java.lang.ref.Reference$Lock)

"main" prio=10 tid=0x00007f5d5c00d000 nid=0xb04 in Object.wait() [0x00007f5d63f61000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x0000000755814468> (a com.sun.jts.CosTransactions.EventSemaphore)
	at java.lang.Object.wait(Object.java:503)
	at com.sun.jts.CosTransactions.EventSemaphore.waitEvent(EventSemaphore.java:133)
	- locked <0x0000000755814468> (a com.sun.jts.CosTransactions.EventSemaphore)
	at com.sun.jts.CosTransactions.RecoveryManager.waitForResync(RecoveryManager.java:1455)
	at com.sun.jts.CosTransactions.TransactionFactoryImpl.localCreate(TransactionFactoryImpl.java:180)
	at com.sun.jts.CosTransactions.CurrentImpl.begin(CurrentImpl.java:428)
	at com.sun.jts.jta.TransactionManagerImpl.begin(TransactionManagerImpl.java:296)
	at com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate.beginJTS(JavaEETransactionManagerJTSDelegate.java:498)
	at com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate.startJTSTx(JavaEETransactionManagerJTSDelegate.java:401)
	at com.sun.enterprise.transaction.JavaEETransactionManagerSimplified.startJTSTx(JavaEETransactionManagerSimplified.java:429)
	at com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate.enlistLAOResource(JavaEETransactionManagerJTSDelegate.java:318)
	at com.sun.enterprise.transaction.JavaEETransactionManagerSimplified.enlistResource(JavaEETransactionManagerSimplified.java:356)
	at com.sun.enterprise.resource.rm.ResourceManagerImpl.registerResource(ResourceManagerImpl.java:152)
	at com.sun.enterprise.resource.rm.ResourceManagerImpl.enlistResource(ResourceManagerImpl.java:112)
	at com.sun.enterprise.resource.pool.PoolManagerImpl.getResource(PoolManagerImpl.java:211)
	at com.sun.enterprise.connectors.ConnectionManagerImpl.getResource(ConnectionManagerImpl.java:354)
	at com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetConnection(ConnectionManagerImpl.java:307)
	at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:196)
	at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:171)
	at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:166)
	at com.sun.messaging.jms.ra.ConnectionFactoryAdapter._allocateConnection(ConnectionFactoryAdapter.java:212)
	at com.sun.messaging.jms.ra.ConnectionFactoryAdapter.createConnection(ConnectionFactoryAdapter.java:170)
	at com.sun.messaging.jms.ra.ConnectionFactoryAdapter.createConnection(ConnectionFactoryAdapter.java:152)
	at com.me.myproject.logging.performance.PerformanceLoggingProducer.log(PerformanceLoggingProducer.java:45)
	at com.me.myproject.logging.performance.PerformanceLoggingProducer$Proxy$_$$_WeldClientProxy.log(Unknown Source)
	at com.me.myproject.ejbinterface.common.RemoteBeanProducer$1.invoke(RemoteBeanProducer.java:128)
	at com.sun.proxy.$Proxy446.getProperty(Unknown Source)
	at com.me.myproject.releasedocuments.basedirectory.BaseDirectoryPropertyInterface.getAppPropertyForBaseDirectory(BaseDirectoryPropertyInterface.java:74)
	at com.me.myproject.releasedocuments.basedirectory.BaseDirectoryPropertyInterface.getBaseDirectory(BaseDirectoryPropertyInterface.java:42)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
	at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
	at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4695)
	at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:630)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
	at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:582)
	at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:46)
	at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
	at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:582)
	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
	at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
	at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4667)
	at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4655)
	at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
	at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
	at com.sun.proxy.$Proxy444.getBaseDirectory(Unknown Source)
	at com.me.myproject.releasedocuments.basedirectory.__EJB31_Generated__BaseDirectoryPropertyInterface__Intf____Bean__.getBaseDirectory(Unknown Source)
	at com.me.myproject.releasedocuments.ReleaseDocumentsInit.initialize(ReleaseDocumentsInit.java:37)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.sun.ejb.containers.interceptors.BeanCallbackInterceptor.intercept(InterceptorManager.java:1035)
	at com.sun.ejb.containers.interceptors.CallbackChainImpl.invokeNext(CallbackChainImpl.java:72)
	at com.sun.ejb.containers.interceptors.CallbackInvocationContext.proceed(CallbackInvocationContext.java:205)
	at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:55)
	at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.sun.ejb.containers.interceptors.CallbackInterceptor.intercept(InterceptorManager.java:986)
	at com.sun.ejb.containers.interceptors.CallbackChainImpl.invokeNext(CallbackChainImpl.java:72)
	at com.sun.ejb.containers.interceptors.CallbackInvocationContext.proceed(CallbackInvocationContext.java:205)
	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.init(SystemInterceptorProxy.java:125)
	at sun.reflect.GeneratedMethodAccessor83.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.sun.ejb.containers.interceptors.CallbackInterceptor.intercept(InterceptorManager.java:986)
	at com.sun.ejb.containers.interceptors.CallbackChainImpl.invokeNext(CallbackChainImpl.java:72)
	at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:412)
	at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:375)
	at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:1949)
	at com.sun.ejb.containers.AbstractSingletonContainer.createSingletonEJB(AbstractSingletonContainer.java:475)
	at com.sun.ejb.containers.AbstractSingletonContainer.access$000(AbstractSingletonContainer.java:81)
	at com.sun.ejb.containers.AbstractSingletonContainer$SingletonContextFactory.create(AbstractSingletonContainer.java:654)
	at com.sun.ejb.containers.AbstractSingletonContainer.instantiateSingletonInstance(AbstractSingletonContainer.java:396)
	- locked <0x000000075b3830d8> (a com.sun.ejb.containers.CMCSingletonContainer)
	at org.glassfish.ejb.startup.SingletonLifeCycleManager.initializeSingleton(SingletonLifeCycleManager.java:219)
	at org.glassfish.ejb.startup.SingletonLifeCycleManager.initializeSingleton(SingletonLifeCycleManager.java:180)
	- locked <0x000000075b383358> (a org.glassfish.ejb.startup.SingletonLifeCycleManager)
	at org.glassfish.ejb.startup.SingletonLifeCycleManager.doStartup(SingletonLifeCycleManager.java:158)
	at org.glassfish.ejb.startup.EjbApplication.start(EjbApplication.java:166)
	at org.glassfish.internal.data.EngineRef.start(EngineRef.java:122)
	at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:291)
	- locked <0x000000075b383478> (a org.glassfish.internal.data.ModuleInfo)
	at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:352)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:497)
	at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:407)
	at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:243)
	at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:281)
	at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:328)
	at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:448)
	at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:163)
	at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2204)
	at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
	- locked <0x000000075003f270> (a java.lang.Object)
	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:673)
	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:660)
	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$UpOneJob.run(CurrentTaskFuture.java:490)
	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$UpAllTheWay.go(CurrentTaskFuture.java:362)
	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$UpAllTheWay.access$100(CurrentTaskFuture.java:279)
	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture.go(CurrentTaskFuture.java:113)
	at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.proceedTo(AsyncRunLevelContext.java:296)
	at org.glassfish.hk2.runlevel.internal.RunLevelControllerImpl.proceedTo(RunLevelControllerImpl.java:66)
	at com.sun.enterprise.v3.server.AppServerStartup.proceedTo(AppServerStartup.java:532)
	at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:329)
	at com.sun.enterprise.v3.server.AppServerStartup.doStart(AppServerStartup.java:226)
	at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:217)
	- locked <0x000000075184dd50> (a com.sun.enterprise.v3.server.AppServerStartup)
	at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79)
	- locked <0x000000075184ddc0> (a com.sun.enterprise.glassfish.bootstrap.GlassFishImpl)
	at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63)
	at com.sun.enterprise.glassfish.bootstrap.osgi.EmbeddedOSGiGlassFishImpl.start(EmbeddedOSGiGlassFishImpl.java:75)
	at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63)
	at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishImpl.start(OSGiGlassFishImpl.java:71)
	at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97)
	at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:54)

"VM Thread" prio=10 tid=0x00007f5d5c0eb800 nid=0xb0e runnable 

"Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f5d5c021800 nid=0xb07 runnable 

"Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f5d5c023800 nid=0xb08 runnable 

"G1 Main Concurrent Mark GC Thread" prio=10 tid=0x00007f5d5c033800 nid=0xb0c runnable 

"Gang worker#0 (G1 Parallel Marking Threads)" prio=10 tid=0x00007f5d5c03d800 nid=0xb0d runnable 

"G1 Concurrent Refinement Thread#0" prio=10 tid=0x00007f5d5c029800 nid=0xb0b runnable 

"G1 Concurrent Refinement Thread#1" prio=10 tid=0x00007f5d5c027800 nid=0xb0a runnable 

"G1 Concurrent Refinement Thread#2" prio=10 tid=0x00007f5d5c026000 nid=0xb09 runnable 


"VM Periodic Task Thread" prio=10 tid=0x00007f5d5c1a0000 nid=0xb16 waiting on condition 

JNI global references: 427


Workaround:

There is a workaround. If the connection and send logic is run in a separate thread, the thread will lock until glassfish is started, then unlock and send the message

@ApplicationScoped
public class PerformanceLoggingProducer {

    private static final Logger logger = Logger.getLogger(PerformanceLoggingProducer.class.getName());

//    @Resource(mappedName = "jms/ConnectionFactory")
    private ConnectionFactory connectionFactory;
//    @Resource(name = "jms/PerformanceLoggingQueue")
    private Queue queue;

    public PerformanceLoggingProducer() {
        try {
            connectionFactory = (ConnectionFactory) new InitialContext().lookup("jms/ConnectionFactory");
            queue = (Queue) new InitialContext().lookup("jms/PerformanceLoggingQueue");
        } catch (NamingException ex) {
            logger.log(Level.SEVERE, null, ex);
        }
    }

    public void log(final PerformanceLogTracker tracker) {

        if (tracker != null) {
      
            Thread t = new Thread(new Runnable() {

                @Override
                public void run() {
                    try (Connection connection = connectionFactory.createConnection()) {
                        Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
                        MessageProducer producer = session.createProducer(queue);
                        ObjectMessage message = session.createObjectMessage(tracker);
                        producer.send(message);
                    } catch (Exception ex) {
                        logger.log(Level.SEVERE, "An error occurred when logging message: " + tracker, ex);
                    }
                }
            });
            
            t.start();

        }
    }

}


 Comments   
Comment by David Zhao [ 05/Mar/14 ]

Is it a timing issue? If sleeping a while (for example, 10 - 60 seconds) before creating connection in the @Singleton @Startup bean helps?





[GLASSFISH-20430] JMS set to Local with start arguments having Xmx results in 2 Xmx args being passed to jvm Created: 29/Apr/13  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: None
Fix Version/s: 4.1

Type: Bug Priority: Major
Reporter: pbelbin Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

jdk 7u17 on solaris and windows


Issue Links:
Dependency
depends on MQ-300 Remove default -Xmx property if it is... Open
Tags: 4_0_1-reviewed, fishcat

 Description   

I have the JMS broker set as 'LOCAL' and have specified the following in the start arguments:

-vmargs -Xmx512m

however, I see the jvm for the broker was started with:

-Xms192m -Xmx192m -Xss192k -Xmx512m

there shouldn't be two Xmx options should there??






[GLASSFISH-19737] Provide default properties in create-jms-resource for admin gui Created: 28/Feb/13  Updated: 28/Feb/13

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: David Zhao Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Currently when creating new jms connection factory via the admin gui, it makes REST calls to create-connector-connection-pool and create-connector-resource separately. It is expected to be changed to call create-jms-resource REST service. But now the REST service of create-jms-resource doesn't return all the default values of create-connector-connection-pool and create-connector-resource. We need find a way to allow it for the admin gui use.






[GLASSFISH-18715] Cannot deny user(s) from producing messages for queues Created: 10/May/12  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1.2
Fix Version/s: 4.1

Type: Bug Priority: Major
Reporter: ashie1287 Assignee: David Zhao
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

RHEL5, OpenMQ 4.5, PostgreSQL as persistent store, OpenLDAP as user repository


Issue Links:
Dependency
depends on GLASSFISH-21082 For GlassFish standalone client witho... Open
depends on MQ-308 accesscontrol: produce.allow with '*'... Resolved
depends on MQ-354 embedded broker: destination accessc... Resolved
depends on MQ-307 username/password parameters in Conne... Closed
Related
is related to MQ-355 embedded broker: does not take global... Resolved
Tags: 4_0_1-reviewed

 Description   

My broker has the following in etc/accesscontrol.properties

###
version=JMQFileAccessControlModel/100
connection.NORMAL.allow.user=*
connection.ADMIN.allow.group=admins

queue..produce.allow.user=
queue..consume.allow.user=

queue.queuename.produce.deny.user=someone
###

What happens:

  • the user 'someone' is allowed to create a producer for queue 'queuename'

What was expected:

  • the user 'someone' should be denied when trying to create a producer for queue 'queuename'

Other notes:

  • if this same type of scenario is repeated for a topic, things work as expected
  • if instead we deny consuming of a queue (e.g. queue.queuename.consume.deny.user=someone) instead of producing of a queue, things work as expected

Seems odd that just this one scenario causes a problem, so it may be worth trying similar ones.



 Comments   
Comment by ashie1287 [ 10/May/12 ]

Did not know that the summary field of the bug uses some kind of markup. Here is what I meant for the accesscontrol.properties file:

###
version=JMQFileAccessControlModel/100
connection.NORMAL.allow.user=*
connection.ADMIN.allow.group=admins

queue.*.produce.allow.user=*
queue.*.consume.allow.user=*

queue.queuename.produce.deny.user=someone
###

Comment by David Zhao [ 06/Jun/13 ]

This partially depends on MQ-307 for applying custom username/password to create connection/producer.

Comment by David Zhao [ 06/Jun/13 ]

Add dependency on MQ-308

Comment by amyk [ 09/Jun/14 ]

Another issue related to this case, discovered by David Zhao, is if a GlassFish JMS connection factory resource is accessed through the following non-standard path, the username/password in ConnectionFactory.createConnection(username, password) doesn't take effect. In this access path, GlassFish connector accesses JMSRA neither in AppClient container nor in EJB/Web container, David, it maybe necessary to require the sign-on information be specified in the JMS resource itself, that is via 'asadmin create-jms-resource --property UserName=username:Password=password' ? - at least that's a workaround.

---------------------------------------
"
Use a standalone GF server with JMS Service of EMBEDDED or LOCAL.
Add a new imq user "imqusermgr add -u myuser -p mypass -g user"
In imq's accesscontrol.properties, add a line "connection.NORMAL.deny.user=guest" to deny guest for getting connection.
Start GF domain
Create JMS connection factory "asadmin create-jms-resource --target server --restype javax.jms.QueueConnectionFactory jms/myFactory ".
Create a JavaSE JMS client application,

Properties jndiProps = new Properties();
jndiProps.put("java.naming.factory.initial", "com.sun.enterprise.naming.impl.SerialInitContextFactory");
jndiProps.put("java.naming.factory.url.pkgs", "com.sun.enterprise.naming");
jndiProps.put("java.naming.factory.state", "com.sun.corba.ee.impl.presentation.rmi.JNDIStateFactoryImpl");
jndiProps.setProperty("org.omg.CORBA.ORBInitialHost", "localhost");
jndiProps.setProperty("org.omg.CORBA.ORBInitialPort", "3700");

ctx = new InitialContext(jndiProps);
qconFactory = (ConnectionFactory) ctx.lookup("jms/myFactory");

Connection qcon = qconFactory.createConnection("myuser", "mypass");
System.out.println(qcon);

Run the client application which is remotely to GF server, then you will get the following security exception that shows the user being authenticated is guest instead of expected myuser which we specified in ConnectionFactory.createConnection(String username, String password) API.

SEVERE: MQJMSRA_MC4001: constructor:Aborting:JMSException on createConnection=[C4084]: User authentication failed: user=guest, broker=localhost:7676(58859), error code: C4084
com.sun.messaging.jms.JMSSecurityException: [C4084]: User authentication failed: user=guest, broker=localhost:7676(58859)
at com.sun.messaging.jmq.jmsclient.ProtocolHandler.authenticate(ProtocolHandler.java:1124)
at com.sun.messaging.jmq.jmsclient.ProtocolHandler.hello(ProtocolHandler.java:1041)
at com.sun.messaging.jmq.jmsclient.ProtocolHandler.hello(ProtocolHandler.java:932)
...
Caused by: com.sun.messaging.jms.JMSSecurityException: [C4084]: User authentication failed: user=guest, broker=localhost:7676(58859)
at com.sun.messaging.jmq.jmsclient.ProtocolHandler.authenticate(ProtocolHandler.java:1124)
at com.sun.messaging.jmq.jmsclient.ProtocolHandler.hello(ProtocolHandler.java:1041)
at com.sun.messaging.jmq.jmsclient.ProtocolHandler.hello(ProtocolHandler.java:932)
at com.sun.messaging.jmq.jmsclient.ConnectionImpl.hello(ConnectionImpl.java:590)
at com.sun.messaging.jmq.jmsclient.ConnectionImpl.openConnection(ConnectionImpl.java:2500)
at com.sun.messaging.jmq.jmsclient.ConnectionImpl.init(ConnectionImpl.java:1156)
at com.sun.messaging.jmq.jmsclient.ConnectionImpl.<init>(ConnectionImpl.java:468)
at com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl.<init>(UnifiedConnectionImpl.java:66)
at com.sun.messaging.jmq.jmsclient.XAConnectionImpl.<init>(XAConnectionImpl.java:64)
at com.sun.messaging.XAConnectionFactory.createXAConnection(XAConnectionFactory.java:110)
at com.sun.messaging.jms.ra.ManagedConnection.<init>(ManagedConnection.java:204)
... 21 more

"

and David Zhao also commented - 04/Jun/14 07:12 AM (copy/paste from MQ-307) - pointing to GlassFish server code

"
The bug is inside ConnectionManagerImpl.internalGetConnection(...), here is the code snippet.

{format}
} else {
if (prin == null) { <--- should it be "if (cxRequestInfo != null)" ? info = new ClientSecurityInfo(cxRequestInfo); } else {
info = new ClientSecurityInfo(prin);
if (prin.equals(defaultPrin)) {{format}

"
---------------------





idletimeoutinseconds accepts value greater than the specified range (GLASSFISH-6914)

[GLASSFISH-18794] Valid value range for jms connection factory resource Idel Timeout attribute Created: 11/Jun/12  Updated: 11/Jun/12

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0_b40
Fix Version/s: None

Type: Sub-task Priority: Major
Reporter: Simon Meng Assignee: shreedhar_ganapathy
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The glassfish document does not mention the valid value range for jms connection factory resource Idle Timeout attribute. The exact value range is [1, 2147483647]

You can create a jms connection factory in following location. Then you can see the Idle Timeout attribute.
http://localhost:4848 --> Common Tasks --> Resources --> JMS Resources --> Connection Factories






Ignoring MDB's activation-spec for RedeliveryInterval and RedeliveryAttempts (GLASSFISH-6617)

[GLASSFISH-18940] Add new property "imq.transaction.message.maxConsecutiveRollbacks" to glassfish jms service Created: 25/Jul/12  Updated: 25/Jul/12

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: None
Fix Version/s: 4.0_b45

Type: Sub-task Priority: Major
Reporter: David Zhao Assignee: Mike Fitch
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Please update glassfish documentation for the enhancement of GLASSFISH-6617:

A new MQ 5.0 property "imq.transaction.message.maxConsecutiveRollbacks" was added.

For the EMBEDDED/LOCAL jms service of glassfish, the property can be set via admin gui:

Configurations -> server-config -> Java Message Service -> Additional Properties

For the REMOTE jms service of glassfish, the property should be set in the remote broker's config properties file.

Please refer to the MQ-134 and MQ-139 for the value of the property:

If <= 0, this feature is disabled;
If > 0, for running in GlassFish with JMSRA, when consecutive rollback a message to a consumer exceeds this number, the broker will automatically put this message to DMQ (if there are more than 1 message in the transaction, only the last message in the transaction is put to DMQ) and and rollback the rest of the transaction then return an error status to rollback






[GLASSFISH-18147] Allow bundled JMS provider JMX mbeans to be available via DAS Created: 07/Jan/12  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1.1, 4.0
Fix Version/s: 4.1

Type: New Feature Priority: Major
Reporter: Sanjeeb Sahoo Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

I don't understand why this has not been done so far. Is it not expected that an administrator should be able to connect to DAS JMX interface and reach all components managed by the same? Bundled JMS provider is one such managed component.



 Comments   
Comment by Sanjeeb Sahoo [ 10/Oct/12 ]

You may like to fix it in some future versions, but that does not change the fact that it applies to GF 3.1.x.





[GLASSFISH-21220] Client cannot connect JMS host having more than one IP address Created: 01/Oct/14  Updated: 17/Oct/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: mkarg Assignee: David Zhao
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows, JDK 7


Attachments: Zip Archive jmsacc.zip    

 Description   

We noticed the following problem at a customer site:

  • Host with two IP addresses is running GF in default configuration.
  • JMS_default_host is set to 0.0.0.0
  • ACC-powered client tries to open JMS session.
  • Client fails and prints on console that it is not possible to connect host 0.0.0.0

Apparently this looks like the server is passing 0.0.0.0 to the client instead of sending the actual IP to which to connect from the particular subnet.

When replacing 0.0.0.0 by the host's actual DNS-registered hostname, a connection from one subnet is working, but not from all others. While DNS correctly resolves to all IPs of that host, it looks like the JMS client in ACC is not using the correct one to connect.



 Comments   
Comment by laps [ 01/Oct/14 ]

We have exactly the same issue with orb-listener (iiop listening process), in the same configuration (one machine, one hostname, two IPs). Glassfish 3.1.2.2. Network address defaulted to 0.0.0.0. Changing this value didn't affect nothing, neither creating a second listener.

Comment by David Zhao [ 09/Oct/14 ]

mkarg,

How did you open JMS session at client side? Could you please share source codes of your ACC-powered client?

David Zhao

Comment by mkarg [ 13/Oct/14 ]

This is legacy code which is rather lenghty, but it boils down to the following:

final ConnectionFactory connectionFactory = DefaultServiceLocator.getInstance().getConnectionFactory("jms/quipsyConnectionFactory");
final Destination destination = DefaultServiceLocator.getInstance().getDestination("jms/quipsyDestination");
this.connection = connectionFactory.createConnection();
final Session session = this.connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
final MessageConsumer consumer = session.createConsumer(destination, this.getSelector(), true);
consumer.setMessageListener(this);
this.connection.start();

The DefaultServiceLocator is using 'new InitialContext().lookup("java:comp/env/")' to find objects by JNDI. The binding is defined by glassfish-application-client.xml which looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE glassfish-application-client PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1 Java EE Application Client 6.0//EN" "http://glassfish.org/dtds/glassfish-application-client_6_0-1.dtd">
<glassfish-application-client>
<resource-ref>
<res-ref-name>jms/quipsyConnectionFactory</res-ref-name>
<jndi-name>QUIPSY_CONNECTION_FACTORY</jndi-name>
</resource-ref>

<resource-env-ref>
<resource-env-ref-name>jms/quipsyDestination</resource-env-ref-name>
<jndi-name>QUIPSY_TOPIC</jndi-name>
</resource-env-ref>
</glassfish-application-client>

Comment by David Zhao [ 14/Oct/14 ]

mkarg,

I wonder what is DefaultServiceLocator? could you please upload your packaged ACC jar and all the source codes for review?

Comment by mkarg [ 15/Oct/14 ]

DefaultServiceLocator is nothing but a wrapper around the code line "new InitialContext().lookup("java:comp/env/")" with implies cast to a particular type. It serves solely for concise code but does not imply any effect. You could just replace it by inlined "(TypeCast) new InitialContext().lookup("java:comp/env/")'"

The code is closed source so I cannot upload it. I also do not see a need for that as the problem happens with any code. GlassFish returns "0.0.0.0" to the ACC in case "0.0.0.0" is specified as the JMS host's IP. That has nothing to do with the application code inside the ACC.

Comment by David Zhao [ 15/Oct/14 ]

mkarg,

I cannot reproduce it with latest GF trunk build. Please see my attached NetBeans project jmsacc which is composed according to your pasted codes. Please check if my reproduce is equivalent to your test case. Can you try my reproducer with the old GF 3.1.1 build to see if you can reproduce it?

My steps:

1. Change the host to "0.0.0.0" for default_JMS_host.
2. Create jms resources.
asadmin --port 4848 create-jms-resource --target server --restype javax.jms.Queue --property imqDestinationName=myQueue jms/myQueue
asadmin --port 4848 create-jms-resource --target server --restype javax.jms.QueueConnectionFactory jms/myFactory
3. Build & deploy jmsacc.jar to glassfish.
4. Get the acc stub
asadmin get-client-stubs --appname=jmsacc d:\work_dir\test\acc
5. Run the acc client
D:\glassfish_trunk\glassfish4\glassfish\bin>appclient -jar d:\work_dir\test\acc\jmsaccClient.jar

Output:
Oct 15, 2014 5:09:50 PM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 5.0.0.Final
Oct 15, 2014 5:09:50 PM com.sun.messaging.jms.ra.ResourceAdapter start
INFO: MQJMSRA_RA1101: GlassFish MQ JMS Resource Adapter: Version: 5.1 (Build 9-b) Compile: July 29 2014 1229
Oct 15, 2014 5:09:50 PM com.sun.messaging.jms.ra.ResourceAdapter start
INFO: MQJMSRA_RA1101: GlassFish MQ JMS Resource Adapter starting: broker is REMOTE, connection mode is TCP
Oct 15, 2014 5:09:50 PM com.sun.messaging.jms.ra.ResourceAdapter start
INFO: MQJMSRA_RA1101: GlassFish MQ JMS Resource Adapter Started:REMOTE
Message sent...
Oct 15, 2014 5:09:50 PM com.sun.messaging.jms.ra.ResourceAdapter stop
INFO: MQJMSRA_RA1101: GlassFish MQ JMS Resource Adapter stopping...
Oct 15, 2014 5:09:50 PM com.sun.messaging.jms.ra.ResourceAdapter stop
INFO: MQJMSRA_RA1101: GlassFish MQ JMS Resource Adapter stopped.
Oct 15, 2014 5:09:50 PM com.sun.enterprise.connectors.service.ResourceAdapterAdminServiceImpl sendStopToResourceAdapter
INFO: RAR7094: jmsra shutdown successful.

Comment by mkarg [ 15/Oct/14 ]

I will try out next week. Meanwhile: Did you really try with client on host A and server on host B, where host B has two LAN adapters in different subnets? That's the actualy case here. It's not about having client and server on the same single-IP host.

Comment by David Zhao [ 17/Oct/14 ]

OK. I will need try to find the hardware resources (2 hosts, and 1 host has two adapters in different subnets) for reproducing this.

Comment by mkarg [ 17/Oct/14 ]

No need for physical hardware: Two virtual machines and a virtual switch will do.





[GLASSFISH-21234] JMS EMBEDDED broker should not start when the port is already used Created: 14/Oct/14  Updated: 15/Oct/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: David Zhao Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to MQ-358 track any integration change necessar... Open

 Description   

This follows up the issue brought up by wiki page http://yakovfain.com/2014/10/09/glassfish-open-mq-and-the-ear-eye-problem/

When the port is already used, the jms embedded broker should not start and it should throw exception and write error messages to GF logs. Thus users will not confuse by the 2 separate brokers which are started manually and by GF embedded.



 Comments   
Comment by amyk [ 14/Oct/14 ]

JMSRA starts EMBEDDED broker with -nobind (due to GlassFish lazy-init JMS service), therefore port 'bind' conflict must be detected, to prevent another broker instance running on the same IP address and port, before GlassFish/JMS module starts the EMBEDDED broker.





[GLASSFISH-21288] SHARED SUBSCRIPTIONS NOT WORKING WITH MDB TOPIC DURABLE SUBSCRIBERS Created: 13/Jan/15  Updated: 13/Jan/15

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1.2.2, 4.0, 4.1
Fix Version/s: future release

Type: Bug Priority: Major
Reporter: cmundt Assignee: Joe Di Pol
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: jms

 Description   

See :GlassFish bug 19568699 - SHARED SUBSCRIPTIONS NOT WORKING WITH MDB TOPIC DURABLE SUBSCRIBERS


When Glassfish is configured to use a remote Broker(s) it is unable to establish a shared durable subscription however when using an EMBEDDED broker it will create a shared subscription fine.

This has been replicated in GF 3.1.2.2, GF 4.0 build 89, and GF Build 13.

Would like to see a fix back filled to GF 4.x if possible.






[GLASSFISH-21286] Reuse jms connection from pool as much as possible when creating new connection in a transaction Created: 12/Jan/15  Updated: 12/Jan/15

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.1
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: David Zhao Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on MQ-360 Reuse jms connection from pool as muc... Open

 Description   

This is a request from customers:

-------- Forwarded Message --------
Subject: [gf-users] JMS Connection pooling
Date: Fri, 9 Jan 2015 10:19:20 -0800
From: Will Hartung <willh@mirthcorp.com>
Reply-To: users@glassfish.java.net
To: users@glassfish.java.net

I was disappointed to learn that when getting a JMS connection more
than once in the same transaction, it actually consumes another
connection from the pool.

We do a lot of side affect event stuff in to JMS, and with a large
transaction, this can burn through connections at a pretty fast rate.

I came up with a way to mitigate it a little bit using a Stateful
Session Bean, and SessionSynchronization. But even with that, in the
end, I have the same issues of having to pass it around, since
whenever I ask for a new bean from JNDI, I get a new bean – and a new
connection. But it does handle connection lifecycle, and closing the
connection properly, in a reasonably elegant manner.

In today's day and age of declarative properties, it sees quite
"creaky" to have to be passing around things like JMS connections in
order to conserve them.

What I would really like is something that lets me tie in to the
current transaction life cycle, then I can use a combination
singleton/threadlocal to cache the "current" JMS connection, and then
clean it up afterward.

I've tried using TransactionSynchronizationRegistry, but things get
out of sync and I get all sorts of transaction errors. I also tried
using TransactionSynchronizationRegistry.getTransactionKey() to use
that as a key to a map, but it too lead to problems. I contemplated
using the UserTranscation as a key, but JNDI won't even give me one in
a CMT environment (which is what I'm in).

I can't just use a raw ThreadLocal, because we do a lot of nested
transactions, so the connection needs to be mated with the current
transaction.

For now, I'm simply passing my Stateful Bean around, just like I did
back 15 years ago. I'm hoping there's a better solution. A GlassFish
specific vs a JEE solution would be fine, we have no plans on porting
this code to anything else anytime soon.

We're running GF 3.1.2.

Speaking of Stateful Beans, if I don't have a @StatefulTimeout, are
the beans automatically destroyed when the transaction
commits/rollsback?

Thanks for any thoughts on this.

Regards,

Will Hartung
(willh@mirthcorp.com)


This message, and any documents attached hereto, may contain confidential
or proprietary information intended only for the use of the addressee(s)
named above or may contain information that is legally privileged. If you
are not the intended addressee, or the person responsible for delivering it
to the intended addressee, you are hereby notified that reading,
disseminating, distributing or copying this message is strictly prohibited.
If you have received this message by mistake, please immediately notify us
by replying to the message and delete the original message and any copies
immediately thereafter. Thank you for your cooperation.






[GLASSFISH-20984] Unable to inject JMS resources in a bean defined in a dependency jar Created: 13/Feb/14  Updated: 20/Apr/15

Status: Open
Project: glassfish
Component/s: cdi, jms
Affects Version/s: 4.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: electricsam Assignee: jjsnyder83
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux, jdk 1.7.0_45



 Description   

I have an @ApplicationScoped bean defined in a common library which I add to my projects in a dependency jar.

Injection of the connection factory and queue results in null objects when using @Resource

Example code:

In this code, I have tried both @Resource(name = "... and @Resource(mappedName = "... with the same result.

@ApplicationScoped
public class PerformanceLoggingProducer {

    private static final Logger logger = Logger.getLogger(PerformanceLoggingProducer.class.getName());

    @Resource(mappedName = "jms/ConnectionFactory")
    private ConnectionFactory connectionFactory;
    @Resource(name = "jms/PerformanceLoggingQueue")
    private Queue queue;

    public void log(final PerformanceLogTracker tracker) {

        if (tracker != null) {
      
            Thread t = new Thread(new Runnable() {

                @Override
                public void run() {
                    try (Connection connection = connectionFactory.createConnection()) {
                        Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
                        MessageProducer producer = session.createProducer(queue);
                        ObjectMessage message = session.createObjectMessage(tracker);
                        producer.send(message);
                    } catch (Exception ex) {
                        logger.log(Level.SEVERE, "An error occurred when logging message: " + tracker, ex);
                    }
                }
            });
            
            t.start();

        }
    }

}

Workaround:

Don't use @Resource for the lookup.

@ApplicationScoped
public class PerformanceLoggingProducer {

    private static final Logger logger = Logger.getLogger(PerformanceLoggingProducer.class.getName());

//    @Resource(mappedName = "jms/ConnectionFactory")
    private ConnectionFactory connectionFactory;
//    @Resource(name = "jms/PerformanceLoggingQueue")
    private Queue queue;

    public PerformanceLoggingProducer() {
        try {
            connectionFactory = (ConnectionFactory) new InitialContext().lookup("jms/ConnectionFactory");
            queue = (Queue) new InitialContext().lookup("jms/PerformanceLoggingQueue");
        } catch (NamingException ex) {
            logger.log(Level.SEVERE, null, ex);
        }
    }

    public void log(final PerformanceLogTracker tracker) {

        if (tracker != null) {
      
            Thread t = new Thread(new Runnable() {

                @Override
                public void run() {
                    try (Connection connection = connectionFactory.createConnection()) {
                        Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
                        MessageProducer producer = session.createProducer(queue);
                        ObjectMessage message = session.createObjectMessage(tracker);
                        producer.send(message);
                    } catch (Exception ex) {
                        logger.log(Level.SEVERE, "An error occurred when logging message: " + tracker, ex);
                    }
                }
            });
            
            t.start();

        }
    }

}


 Comments   
Comment by jjsnyder83 [ 20/Apr/15 ]

Can you provide a test app? I'd like to see how it's packaged.





asadmin create-jmsdest properties not applied to EMBEDDED MQ broker (GLASSFISH-20378)

[GLASSFISH-20405] Docs: Need update the property names of create-jmsdest subcommand Created: 25/Apr/13  Updated: 29/Apr/15

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 3.1.2
Fix Version/s: 4.0

Type: Sub-task Priority: Major
Reporter: David Zhao Assignee: Mike Fitch
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The property names are incorrect in document http://docs.oracle.com/cd/E26576_01/doc.312/e24938/create-jmsdest.htm that the property names should be upper case for the first character.

For more details, please see the one pager by running "asadmin create-jmsdest --help".



 Comments   
Comment by sergeich [ 29/Apr/15 ]

Output of the command "asadmin create-jmsdest --help" in Glassfish 4.1 also contains the same error (names of properties start with lower-case letter).





[GLASSFISH-7118] If JMSRA returns isSameRM=true then TM calls incorrect sequence of start() and end() Created: 02/Feb/09  Updated: 08/Dec/10

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: v2.1
Fix Version/s: future release

Type: Bug Priority: Minor
Reporter: Nigel Deakin Assignee: marina vatkina
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 7,118
Status Whiteboard:

V2.1.1_exclude


 Description   

I have discovered that if I change the implementation of XAResource in JMSRA to
return isSameRM()=true then the GlassFish TM does not perform the correct
sequence of calls to start() and end().

If I make this change, I observe the following sequence of calls when a MDB
running in GlassFish 2.1 receives a message and sends a second message, within
an XA transaction.

(inboundXAResource is the XAResource for the inbound message; outboundXAResource
is the XAResource for the outbound message)

1. inboundXAResource.start(xid1, TMNOFLAGS)

2. outboundXAResource.start(xid1, TMJOIN)
3. work done using outbound resource (sending the outbound message to the broker)
4. outboundXAResource.end(xid1, TMSUCCESS)

5. work done using inbound resource (sending an ack back to the broker)
6. inboundXAResource.end(xid1, TMSUCCESS)
7. transaction is committed (details omitted)

Step (4) is incorrect: it should not end() the transaction branch until just
before the commit.

Here is more information about how I came to this conclusion:

Steps 2,3 and 4 were triggered by the MDB performing the following:

connection = getOutboundConnectionFactory().createConnection();
session = connection.createSession(false,
javax.jms.Session.AUTO_ACKNOWLEDGE);
TextMessage tm = session.createTextMessage(text);
MessageProducer messageProducer =
session.createProducer(getOutboundDestination());
messageProducer.setDeliveryMode(deliveryMode);
messageProducer.send(tm);
session.close();
connection.close();

...which is all perfectly standard and correct code.

When createConnection() is called, this causes outboundXAResource.start(xid1,
TMJOIN) (step 2) to be called:

XAResourceForMC.start(Xid, int) line: 880
TransactionState.startAssociation(XAResource, Control, int) line: 312
TransactionImpl.enlistResource(XAResource) line: 205
J2EETransaction.enlistResource(XAResource) line: 577

J2EETransactionManagerOpt(J2EETransactionManagerImpl).enlistResource(Transaction, ResourceHandle)
line: 372
J2EETransactionManagerOpt.enlistResource(Transaction, ResourceHandle) line:
144
ResourceManagerImpl.registerResource(ResourceHandle) line: 144
ResourceManagerImpl.enlistResource(ResourceHandle) line: 102
PoolManagerImpl.getResource(ResourceSpec, ResourceAllocator,
ClientSecurityInfo) line: 216
ConnectionManagerImpl.internalGetConnection(...) line: 337
ConnectionManagerImpl.allocateConnection(...) line: 235
ConnectionManagerImpl.allocateConnection(ManagedConnectionFactory,
ConnectionRequestInfo, String) line: 165
ConnectionManagerImpl.allocateConnection(ManagedConnectionFactory,
ConnectionRequestInfo) line: 158
ConnectionFactoryAdapter._allocateConnection(String, String) line: 179
ConnectionFactoryAdapter.createConnection(String, String) line: 166
ConnectionFactoryAdapter.createConnection() line: 148
MessageForwarderBean(GenericMDB).sendJMSMessageToOutboundQueue(String) line:
71
MessageForwarderBean(GenericMDB).onMessage(Message) line: 56

When connection.close() is called, this causes outboundXAResource.end(xid1,
TMSUCCESS) (step 4) to be called:

XAResourceForMC.end(Xid, int) line: 468
TransactionState.endAssociation(XAResource, int) line: 359
TransactionImpl.delistResource(XAResource, int) line: 242
J2EETransaction.delistResource(XAResource, int) line: 563

J2EETransactionManagerOpt(J2EETransactionManagerImpl).delistResource(Transaction, ResourceHandle,
int) line: 832
J2EETransactionManagerOpt.delistResource(Transaction, ResourceHandle, int)
line: 229
ResourceManagerImpl.unregisterResource(ResourceHandle, int) line: 265
ResourceManagerImpl.delistResource(ResourceHandle, int) line: 223
PoolManagerImpl.resourceClosed(ResourceHandle) line: 400
ConnectorAllocator$ConnectionListenerImpl.connectionClosed(ConnectionEvent)
line: 72
ConnectionEventListener.sendEvent(int, Exception, Object) line: 143
ManagedConnection.sendEvent(int, Exception, Object) line: 676
ConnectionAdapter.close() line: 255
MessageForwarderBean(GenericMDB).sendJMSMessageToOutboundQueue(String) line:
84
MessageForwarderBean(GenericMDB).onMessage(Message) line: 56

However we are still in onMessage() here, and so I don't think it is correct to
call end() at this point. I think this is the root cause of the problem.

I have performed the same test using the JMSJCA resource adaptor (which seems to
work fine with a JOIN), and I notice that in this case the call to
outboundXAResource.end(xid1, TMSUCCESS) is deferred until just before the commit,

Using the debugger I have compared the code that is executed to try to find out
why JMSJCA seems to "work" whilst JMSRA doesn't.

The reason can be found in J2EETransactionManagerImpl (part of GlassFish)

public boolean delistResource(Transaction tran, ResourceHandle h,
int flag)
throws IllegalStateException, SystemException {
if (_logger.isLoggable(Level.FINE))
_logger.log(Level.FINE,"TM: delistResource");
if (!h.isShareable() || multipleEnlistDelists) {
if (h.isTransactional() && h.isEnlisted())

{ return tran.delistResource(h.getXAResource(), flag); }

else

{ return true; }

}
return true;
}

As can be seen, if the resource handle is "shareable" the resource is not
delisted. In GlassFish, the JMSRA is hard-coded to be non-sharable whilst JMSJCA
is not, which is why the two behave differently.

As far as I can see the fact that this works at all in JMSJCA is a fluke, since
according to Frank Kieviet a JMSJCA resource is not actually shareable, and the
SeeBeyond integration server had it hardcoded as such. However this was never
done in GlassFish.

Nevertheless it seems that this shareable flag seems to be the reason why
isSameRM() works in JMSJCA but not with JMSRA.

However I don't think that setting shareable to false is the correct solution,
since (as I said) the concept of connection sharing and the concept of joining
resources in a transaction branch being different and concepts: just because
isSameRM() returns true doesn't mean that JMSRA now support connection sharing.

In particular, I am concerned that enabling connection sharing with JMSRA may
have undesirable side effects since it may cause the
application server to reuse connection objects in a way that it did not do
before, and which might not work.

There are numerous places in GlassFish which hard-code "shareable" to be false
for the JMSRA resource adaptor. I don't
personally know why this was done, but I have heard suggestions that it was done
to pass the JMS CTS tests.



 Comments   
Comment by marina vatkina [ 11/Jun/09 ]

Reassigning to jms to followup.

From Sankar's email:

IMO, things should work once we make Resourcehandle.isShreable() return true for
JMS Resource also (the way it is done for the JDBC Resource) and I believe, this
is the least risk option. I don't think just because
Resourcehandle.isShreable() is returned true (Ramesh gave a patch for the same
some time ago), we are reusing the same inbound connection for outbound
connection and vice versa. I would expect at least we test once the isSameRM()
is implemented properly, using the Ramesh's patch, whether thing would work or not.

Comment by Nigel Deakin [ 19/Aug/09 ]

As I feared, changing Glassfish to perform connection sharing when using JMSRA
has undesired side-effects: See Glassfish 2.1.1 issue 8997. In short, JMSRA does
not support connection sharing, and so simply emabling its use is not sufficient
to resolve this issue.

Comment by jthoennes [ 20/Aug/09 ]

Will these changes part of OpenMQ 4.4 and GF v2.1.1?

Actually, I am looking forward to the integration of the Generic JMSRA and the
JMSJCA into OpenMQ 4.4.

Comment by fkieviet [ 24/Aug/09 ]

add fkieviet to cc

Comment by kumara [ 01/Sep/09 ]

Changing version from 9.1.1 to v2.1 to reflect new name/version.

Comment by Satish Kumar [ 07/Sep/09 ]

Reassiging the issue to Marina since this seems to be more of a TM issue than
JMS related.

Comment by jagadesh [ 15/Oct/09 ]

Will not be fixed for V2.1.1

Comment by jagadesh [ 15/Oct/09 ]

Will not be fixed for V2.1.1





[GLASSFISH-5523] JMS - Glass fish example Created: 18/Aug/08  Updated: 06/Mar/12

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 9.0pe
Fix Version/s: not determined

Type: Bug Priority: Minor
Reporter: telukuntla Assignee: akira_ag
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 5,523

 Description   

Hi ,

Currently i am working with JBoss with JMS and working fine. but i want to
change the server JBoss to Glass Fish.
i dont know any thing abt Glass fish.
Please let provide me the some sample example to create Topics and sending /
receiving messges.

and also how to create topic dynamically in glass fish server.

Please give me the solution ASAP.

thaks,
Raaj



 Comments   
Comment by sanandal [ 11/Jan/09 ]

"Reclassifying as P4 because this issue is not deemed "must fix" for this v2.1
release whose primary release driver is SailFin.
This issue will be scrubbed after this release and will be given the right
priority for the next release."

Comment by Tom Mueller [ 06/Mar/12 ]

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.





[GLASSFISH-5316] Exception : jmsdirect failed for connectionId Created: 16/Jul/08  Updated: 06/Mar/12

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 9.1peur2
Fix Version/s: not determined

Type: Bug Priority: Minor
Reporter: rahulbiswas Assignee: rahulbiswas
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 5,316

 Description   

I am running into the following exception with the embedded broker on Solaris 10
and GF 9.1_02 when using an EMBEDDED broker. Don't see this on a remote broker.
Since the application is a benchmark, would appreciate a fix as soon as possible
as using a remote broker would not allow us to scale.

[#|2008-06-26T21:54:23.078-0700|SEVERE|sun-appserver9.1|javax.resourceadapter.mqjmsra.outbound.connection|_ThreadID=17;_ThreadName=httpSSLWorkerThread-8000-0;_RequestID=1fc42f4a-7ce7-4f01-8b41-ad2b8bb36eb8;|prepareTransaction
(XA) on JMSService:jmsdirect failed for connectionId:2859252261286262016 due to
TxID is already in use.|#]

[#|2008-06-26T21:54:23.079-0700|WARNING|sun-appserver9.1|javax.enterprise.system.core.transaction|_ThreadID=17;_ThreadName=httpSSLWorkerThread-8000-0;java.lang.RuntimeException:
javax.transaction.xa.XAException;prepare;_RequestID=1fc42f4a-7ce7-4f01-8b41-ad2b8bb36eb8;|JTS5031:
Exception [java.lang.RuntimeException: javax.transaction.xa.XAException] on
Resource [prepare] operation.|#]

[#|2008-06-26T21:54:23.085-0700|SEVERE|sun-appserver9.1|com.sun.xml.ws.server.sei.EndpointMethodHandler|_ThreadID=17;_ThreadName=httpSSLWorkerThread-8000-0;_RequestID=1fc42f4a-7ce7-4f01-8b41-ad2b8bb36eb8;|Transaction
aborted; nested exception is: javax.transaction.RollbackException
javax.ejb.EJBException: Transaction aborted; nested exception is:
javax.transaction.RollbackException
javax.transaction.RollbackException
at
com.sun.jts.jta.TransactionManagerImpl.commit(TransactionManagerImpl.java:311)
at
com.sun.enterprise.distributedtx.J2EETransactionManagerImpl.commit(J2EETransactionManagerImpl.java:1030)
at
com.sun.enterprise.distributedtx.J2EETransactionManagerOpt.commit(J2EETransactionManagerOpt.java:397)
at com.sun.ejb.containers.BaseContainer.completeNewTx(BaseContainer.java:3792)
at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:3571)
at
com.sun.ejb.containers.WebServiceInvocationHandler.invoke(WebServiceInvocationHandler.java:200)
at $Proxy84.checkForRequestedWork(Unknown Source)
at sun.reflect.GeneratedMethodAccessor168.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)



 Comments   
Comment by Sivakumar Thyagarajan [ 23/Jul/08 ]

Requesting Satish to investigate and fix.

Comment by harpreet [ 04/Sep/08 ]

Marking target milestone as 9.1.1.

Comment by Satish Kumar [ 08/Sep/08 ]

Documented workaround -
try setting the system property -
'com.sun.enterprise.connectors.system.RevertToEmbedded' to 'true'

Also, need to know if this issue is also occuring in the LOCAL mode.

Comment by rahulbiswas [ 25/Sep/08 ]

With the workaround suggested, I am getting a different exception in the
EMBEDDED mode.

[#|2008-09-25T14:02:27.904-0700|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=21;_ThreadName=Session1527877626908213760;|
DirectConsumer:Caught Exception delivering
messagecom.sun.messaging.jmq.io.Packet cannot be cast to
com.sun.messaging.jms.ra.DirectPacket|#]

Comment by aef [ 21/Nov/08 ]

Maybe related with Issue 5401?

https://glassfish.dev.java.net/issues/show_bug.cgi?id=5401

Comment by sanandal [ 11/Jan/09 ]

"Reclassifying as P4 because this issue is not deemed "must fix" for this v2.1
release whose primary release driver is SailFin.
This issue will be scrubbed after this release and will be given the right
priority for the next release."

Comment by Tom Mueller [ 06/Mar/12 ]

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.





[GLASSFISH-5214] Destination not serializable Created: 25/Jun/08  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 9.1peur2
Fix Version/s: 4.1

Type: Bug Priority: Minor
Reporter: cplaetzinger Assignee: Nigel Deakin
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Linux


Issuezilla Id: 5,214
Status Whiteboard:

as911-na


 Description   

Hello,

i try to bind JMS destination into JNDI. This will throw the following exception:

java.io.NotSerializableException: com.sun.messaging.jmq.jmsservice.Destination
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1156)
	at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
	at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
	at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
	at com.sun.enterprise.naming.NamingUtils.makeCopyOfObject(NamingUtils.java:64)
	at
com.sun.enterprise.naming.LocalSerialContextProviderImpl.bind(LocalSerialContextProviderImpl.java:89)
	at com.sun.enterprise.naming.SerialContext.bind(SerialContext.java:461)
	at javax.naming.InitialContext.bind(InitialContext.java:400)
	at
com.macd.research.temporaryqueues.MessageRegistrar.onMessage(MessageRegistrar.java:65)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
com.sun.enterprise.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1067)
	at com.sun.enterprise.security.SecurityUtil.invoke(SecurityUtil.java:176)
	at
com.sun.ejb.containers.BaseContainer.invokeTargetBeanMethod(BaseContainer.java:2895)
	at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:3986)
	at
com.sun.ejb.containers.MessageBeanContainer.deliverMessage(MessageBeanContainer.java:1111)
	at
com.sun.ejb.containers.MessageBeanListenerImpl.deliverMessage(MessageBeanListenerImpl.java:74)
	at
com.sun.enterprise.connectors.inflow.MessageEndpointInvocationHandler.invoke(MessageEndpointInvocationHandler.java:179)
	at $Proxy58.onMessage(Unknown Source)
	at com.sun.messaging.jms.ra.OnMessageRunner.run(OnMessageRunner.java:258)
	at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:76)
	at
com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:555)

Example code:

	public void onMessage(Message incomingMessage) {
		InitialContext ic;
		try {
			ic = new InitialContext();
			String jndiName = "macd/research/ReplyDestination";
			Destination replyDestination = incomingMessage.getJMSReplyTo();
			if (replyDestination instanceof Serializable) {
				System.out.println("Destination is serializable!");
			} else {
				System.out.println("Destination is NOT serializable!");
			}
			ic.bind(jndiName + "Test", replyDestination);
		} catch (NamingException e) {
			// ignore for testing
		} catch (JMSException e) {
			// ignore for testing
		}
	}

But the instanceof operation returns true. So the bind() method tries to
serialize the given object which cause mentioned exception. This happens if JMS
service type is set to "embedded". If i switch the type to "local" everything
works fine. In my opinion there is something wrong . The instanceof operation
should return false if the object is not serializable.



 Comments   
Comment by Sivakumar Thyagarajan [ 23/Jul/08 ]

Requesting Satish to investigate and fix.

Comment by chrjohn [ 23/Jul/08 ]
      • Issue 5214 has been confirmed by votes. ***
Comment by harpreet [ 04/Sep/08 ]

Marking target milestone as 9.1.1.

Comment by Satish Kumar [ 24/Sep/08 ]

Investigating the issue...

Comment by harpreet [ 20/Oct/08 ]

Please scrub issue and see if it is critical to v2.1.

Comment by Satish Kumar [ 29/Oct/08 ]

Downgrading the priority since the problem only exists in the EMBEDDED mode.
This appears to be an issue with the MQ. Will target a fix for the next version
of MQ (4.4).

Comment by sanandal [ 11/Jan/09 ]

"Reclassifying as P4 because this issue is not deemed "must fix" for this v2.1
release whose primary release driver is SailFin.
This issue will be scrubbed after this release and will be given the right
priority for the next release."

Comment by David Zhao [ 14/Feb/12 ]

The reason is that although com.sun.messaging.jms.ra.TemporaryQueue is serializable, but it references com.sun.messaging.jmq.jmsservice.Destination which is not serializable.

Comment by Tom Mueller [ 06/Mar/12 ]

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.

Comment by David Zhao [ 29/May/12 ]

Nigel,

com.sun.messaging.jms.ra.TemporaryQueue is serializable which is extended from com.sun.messaging.jmq.jmsclient.resources.ClientResources.AdministeredObject, but it indirectly references com.sun.messaging.jmq.jmsservice.Destination (from within its parent class of com.sun.messaging.jms.ra.TemporaryDestination) which is not serializable.





[GLASSFISH-7717] Wrong handling of RuntimeExceptions in JMS cluster Created: 14/Apr/09  Updated: 02/Nov/12

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 9.1peur2
Fix Version/s: 4.0

Type: Bug Priority: Minor
Reporter: friedeks Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Linux
Platform: All


Issuezilla Id: 7,717
Status Whiteboard:

v3_exclude, v2.1.1_exclude

Tags: bj-reviewed

 Description   

I installed a v2.1 cluster with two nodes and set up a default jms
ConnectionFactory and logical + physical queue.
I composed a testcase with one MDB doing nothing but throwing a RuntimeException.
The MDB was configured with "EndpointExceptionRedeliveryAttempts=5" and deployed
to the cluster.
A JavaSE Client uses JNDI to send 10000 transactional messages to the queue.
The load gets distributed evenly between the two cluster instances. Instance one
behaves as expected and retries 5 times before moving the messages to DMQ.
Instance two instead seems to ignore EndpointExceptionRedeliveryAttempts and
keeps trying to deliver the bad messages. Instance two loops forever using a lot
of CPU. The Queue shows around 5000 "Not ACK" messages.

The same testcase runs flawlessly in a non-clustered environment.



 Comments   
Comment by Satish Kumar [ 09/Sep/09 ]

This is a v2.1 issue not relevant for V3. Hence adding 'v3_exclude' in the
status whiteboard.

Comment by Ed Bratt [ 15/Oct/09 ]

Will not fix in v2.1.1

Comment by Tom Mueller [ 06/Mar/12 ]

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.

Comment by Simon Meng [ 24/Oct/12 ]

I can't reproduce the described issue. The expected RuntimeException appeared. There is no infinite loop.

But I found other issue in my testing. When sending a message from the standalone java client to queue, the MDB behaves as expected and retries 5 times, we can see 6 RuntimeExceptions. Then the message was moved to DMQ. It looks everything works fine at this point. But when I restart domain and server, the MDB throws 6 RuntimeExceptions again. The message also enters queue and marked as "UnAck". Each time I retart domain, the MDB still throws 6 RuntimeExceptions. It looks to me the behavior is not correct. The message has been moved to the DMQ, why it is delivered again and again when domain restart?

Comment by Simon Meng [ 02/Nov/12 ]

The "other issue" that simeng_oracle noticed in his testing as mentioned in his above comment - that involves a transacted sender client and restart domain is a MQ bug MQ-227.





[GLASSFISH-4159] Failed to look up jms resources from other web servers Created: 12/Feb/08  Updated: 06/Jan/11

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 9.1peur1
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Cheng Fang Assignee: Jagadish
Resolution: Unresolved Votes: 4
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 4,159

 Description   

When looking up glassfish jms resources from other web servers like tomcat,
jetty, or resin, got the following exception:

com.sun.enterprise.connectors.ConnectorRuntimeException: Failed to look up
ConnectorDescriptor from JNDI
com.sun.enterprise.naming.factory.ConnectorObjectFactory.getObjectInstance(ConnectorObjectFactory.java:98)
javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:304)
com.sun.enterprise.naming.SerialContext.lookup(SerialContext.java:403)
javax.naming.InitialContext.lookup(InitialContext.java:351)
foo.FooServlet.processRequest(FooServlet.java:56)
foo.FooServlet.doGet(FooServlet.java:95)
javax.servlet.http.HttpServlet.service(HttpServlet.java:718)
javax.servlet.http.HttpServlet.service(HttpServlet.java:831)

All the necessary glassfish jar files have been copied to tomcat:
appserv-deployment-client.jar
appserv-admin.jar
javaee.jar
zappserv-rt.jar
appserv-ext.jar
imqjmsra.jar

Others have also reported the exactly the same problem on glassfish forum, with
no solution yet:

JMS remote client
http://forums.java.net/jive/thread.jspa?messageID=255213

JMS lookup problem
http://forums.java.net/jive/thread.jspa?threadID=36502&tstart=0



 Comments   
Comment by Cheng Fang [ 12/Feb/08 ]

In ConnectorObjectFactory.getObjectInstance(...), it seems the wrong
InitialContext is instantiated. Here we always call the no-arg constructor of
InitialContext(), which will result in the Tomcat naming manager if run inside
tomcat. We should use the nameCtx param passed in. nameCtx is currently never
referenced in this method.

That may also explains why the same code would work in a standalone client
(where only one naming manager, the default one, is available), but not from a
foreign web container.

public class ConnectorObjectFactory implements ObjectFactory {

public Object getObjectInstance(Object obj,
Name name,
Context nameCtx,
Hashtable env) throws Exception
{
Reference ref = (Reference) obj;
if(_logger.isLoggable(Level.FINE))

{ _logger.log(Level.FINE,"ConnectorObjectFactory: " + ref + " Name:" + name); }

String poolName = (String) ref.get(0).getContent();
String moduleName = (String) ref.get(1).getContent();

Switch sw = Switch.getSwitch();

if(runtime.getEnviron() == ConnectorRuntime.CLIENT) {
ConnectorDescriptor connectorDescriptor = null;
try

{ Context ic = new InitialContext(); String descriptorJNDIName = ConnectorAdminServiceUtils. getReservePrefixedJNDINameForDescriptor(moduleName); connectorDescriptor = (ConnectorDescriptor)ic.lookup(descriptorJNDIName); }

catch(NamingException ne)

{ _logger.log(Level.FINE, "Failed to look up ConnectorDescriptor from JNDI", moduleName); throw new ConnectorRuntimeException( "Failed to look up ConnectorDescriptor from JNDI"); }
Comment by Sivakumar Thyagarajan [ 04/Apr/08 ]

Requesting Jagadish to investigate

Comment by harpreet [ 09/Apr/08 ]

Not critical for v2.1 marking for next release.

Comment by travisb [ 08/Sep/08 ]
      • Issue 4159 has been confirmed by votes. ***
Comment by sanandal [ 11/Jan/09 ]

"Reclassifying as P4 because this issue is not deemed "must fix" for this v2.1
release whose primary release driver is SailFin.
This issue will be scrubbed after this release and will be given the right
priority for the next release."

Comment by Jagadish [ 29/Jan/10 ]

Fixed in v3.1 (trunk)

svn log -v -r 35516

https://glassfish-svn.dev.java.net/servlets/ReadMsg?list=commits&msgNo=18049





[GLASSFISH-15722] mq scripts to work on windows Created: 27/Jan/11  Updated: 02/Dec/11

Status: Reopened
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: sonialiu Assignee: Jill Sato
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

windows



 Description   

I got the following error when running ha/mq test on windows (BTW, we are using cygwin on window to run HA test).

[testng] INFO: Command Executed at agent machine agent2: sh cygwin-wrapper C:/ha/glassfish3/mq/bin/imqdbmgr recreate tbl -Dimq.brokerid=stclusterinstance103 -Dimq.cluster.url=file:C:/ha/file_repository/mq-test-properties/cluster.properties
[testng] Output :
[testng] Jan 27, 2011 2:49:32 PM com.sun.dft.glassfish.utils.Utility logCommandOutput
[testng] SEVERE: C:/ha/glassfish3/mq/bin/../etc/imqenv.conf: line 41: $'\r': command not found
[testng] C:/ha/glassfish3/mq/bin/../etc/imqenv.conf: line 45: $'\r': command not found
[testng] C:/ha/glassfish3/mq/bin/../etc/imqenv.conf: line 49: $'\r': command not found
[testng] java.lang.NoClassDefFoundError: com/sun/messaging/jmq/jmsserver/persist/jdbc/DBTool
[testng] Caused by: java.lang.ClassNotFoundException: com.sun.messaging.jmq.jmsserver.persist.jdbc.DBTool
[testng] at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
[testng] at java.security.AccessController.doPrivileged(Native Method)
[testng] at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
[testng] at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
[testng] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
[testng] at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
[testng] Could not find the main class: com.sun.messaging.jmq.jmsserver.persist.jdbc.DBTool. Program will exit.
[testng] Exception in thread "main"

I checked the imqenv.conf, the line 41, 45, 49 which the error complained are blank lines. I checked with Amy, she thinks we should put "#" in front of those lines.



 Comments   
Comment by sonialiu [ 27/Jan/11 ]

All other mq commands like imqcmd, imqbrokerd... also failed with the similar error. This affected the ha/mq test run on windows. We have to have a fix for this.

Comment by Satish Kumar [ 27/Jan/11 ]

Requesting Amy to have a look at this issue...

Comment by amyk [ 28/Jan/11 ]

Sonia, it looks like you were using the script version of MQ commands which might be there for platform independent packaging. On Windows, please use the .exe version

reasign to Jill to decide if the script version of MQ commands on Windows should doc or change packaging

Comment by Jill Sato [ 28/Jan/11 ]

It is documented that on windows, you need to use the imq*.exe's. The scripts are for unix systems. Closing issue.

Comment by Jill Sato [ 28/Jan/11 ]

Not a bug.
Must uses mq*.exe exeutables on windows.

Comment by kumara [ 28/Jan/11 ]

Most of the other glassfish scripts work on both Windows (with MKS or Cygwin) and Unix. It is best to have consistent behavior in all scripts included with GlassFish. I would recommend reopening and converting this to an enhancement for the next release.

Comment by Jill Sato [ 28/Jan/11 ]

Snjezana said a couple of scripts works on windows, but not all.
We can try to convert mq scripts to work on windows. Will reopen for next release.
Strange that it would require a shell on win.

Comment by Jill Sato [ 28/Jan/11 ]

reopen for next release

Comment by Snjezana Sevo-Zenzerovic [ 28/Jan/11 ]

See issue GLASSFISH-6974 for more information on the overall Cygwin support status of GlassFish scripts.





[GLASSFISH-19044] JMS destination without physical destination name can be created Created: 31/Aug/12  Updated: 01/Jul/13

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0_b45
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: tak09 Assignee: Anissa Lam
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

JMS destination without physical destination name can be created. To reproduce the problem, follow the steps below.

Create a new JMS destination resource

1.Open Resources > JMS Resources > Destination Resources from the left side Navigation tree.
2.Click New.
3.Enter all the mandatory fields including "Physical Destination Name".
4.Click OK to save.

Then, remove the Name property.

1.Open the newly created JMS destination resource from the Navigation menu Resources > Connectors > Admin Object Resources.
2.Click Delete property Delete the Name property.
3.Then, click OK to save it.

Go back to the JMS destination resource.

1.Open the same JMS destination resource from Resources > JMS Resources > Destination Resources from the left side Navigation tree.
2.You will find the Physical Destination Name which is a mandatory field became empty.
(Bug)This field is not supposed to become empty.



 Comments   
Comment by David Zhao [ 01/Jul/13 ]

Anissa,

Could you please evaluate if there is any solution for the case?

/David

Comment by Anissa Lam [ 01/Jul/13 ]

In the console, I can give error and stop user from deleting the Name Property in the Admin Object Resource page.
However, this is not going to stop user using CLI or REST resource to remove that property.

So, the ideal is to stop the removal of this Name Property in the Admin Object config code, and returns an error if user wants to remove it, independent of the client.





[GLASSFISH-19037] Invalid characters can be entered for Name in New Admin Object Resource screen Created: 27/Aug/12  Updated: 04/Jun/14

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0_b45
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: tak09 Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows


Attachments: PNG File New Admin Object Resource.png    
Tags: 4_0_1-reviewed

 Description   

In the New Admin Object Resource screen, an invalid characters can be entered for name property.

To reproduce the issue:
1. From the GUI, open Resources > Connectors > Admin Object Resources.
2. Then, click New to open the 'New Admin Object Resource'.
3. Enter an invalid character in the Name property. For example !abc. Please see the screenshot.
4. When you click OK, it is created with the invalid property name.

According to the imqcmd manual, the valid characters for the physical destination name are as follows:

"The destination name destName may contain only alphanumeric characters (no spaces) and must begin with an alphabetic character or the underscore (_) or dollar sign ($) character. It may not begin with the characters mq."

http://docs.oracle.com/cd/E26576_01/doc.312/e24943/command-line-reference.htm#aeonj

Please see the related issue GLASSFISH-19033 as well.



 Comments   
Comment by David Zhao [ 09/Jul/13 ]

Forward it to connector team for further investigation about the create-admin-object command issue.

Comment by Jagadish [ 18/Apr/14 ]

I think JMS RA need to validate the value of the property being set.
eg: the method "setName" in the use-case stated must do validation and throw exception in case the value is not valid.
(You can do validation in the setName or use bean validation). Transferring to JMS team for the fix.





[GLASSFISH-21292] If messages are put in DMQ and Glassfish is restarted, the messages get consumed twice Created: 21/Jan/15  Updated: 21/Jan/15

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: FionaS Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

linux



 Description   

I had this problem with Glassfish 4, which was using

Message Queue 5.0 Oracle Version: 5.0 (Build 14-e) Compile: April 12 2013 0104

I had a really simple experimental application, similar to the app from https://weblogs.java.net/blog/felipegaucho/archive/2009/09/24/handling-poison-messages-glassfish.

The onMessage() called a SLB method which had a TransactionAttributeType.REQUIRES_NEW. ( and did some simple JPA stuff)

If the messages temporarily could not be consumed, they were put in the DMQ.
If I stopped/started Glassfish, they were read from the DMQ and
processed TWICE. I ended up with twice the JPA entries I should have!

Anyway, I found someone who had the same problem in Stackoverflow
with GF 3.1.2
http://stackoverflow.com/questions/16716087/strange-behavior-of-jms-queue-in-glassfish-3-1-2






[GLASSFISH-20866] jmsra.upgrade_check_failed WARNING on Glassfish startup Created: 23/Oct/13  Updated: 23/Oct/13

Status: Open
Project: glassfish
Component/s: jms
Affects Version/s: 4.0
Fix Version/s: None

Type: Bug Priority: Trivial
Reporter: Andrew_Scully Assignee: David Zhao
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Glassfish 4.0 with embedded OpenMQ


Issue Links:
Duplicate
is duplicated by MQ-346 jmsra.upgrade_check_failed WARNING on... Closed
Tags: glassfish4, jms, log, mq, ra, startup, warning

 Description   

[Originally raised against MQ: https://java.net/jira/browse/MQ-346]

During startup, the following entry is written to the log...

2013-10-22 15:27:12,442 WARNING glassfish 4.0 javax.enterprise.resource.resourceadapter.com.sun.enterprise.connectors.jms.util _ThreadID=15;_ThreadName=WrapperSimpleAppMain; jmsra.upgrade_check_failed

...despite this, all of the JMS functionality seems to work fine.

This post...

https://forums.oracle.com/thread/1520247

...appears to suggest that the WARN level is inappropriate.






Generated at Thu May 28 01:55:43 UTC 2015 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.