glassfish
  1. glassfish
  2. GLASSFISH-20983

Deadlock when creating a JMS connection when Glassfish is starting

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 4.0
    • Fix Version/s: None
    • Component/s: jms
    • Labels:
      None
    • Environment:

      Linux, jdk 1.7.0_45

      Description

      Note: This is not reproducible in a standalone environment using the embedded broker.

      In my environment I have a cluster with 2 nodes, using enhanced local brokers. I have also tried enhanced remote brokers with the same effect.

      When glassfish is starting, but not completed, if an application tries to send a message deadlock occurs when creating the JMS connection. This can easily be reproduced by trying to send a message in a @Singleton @Startup bean.

      Producer:

      @ApplicationScoped
      public class PerformanceLoggingProducer {
      
          private static final Logger logger = Logger.getLogger(PerformanceLoggingProducer.class.getName());
      
      //    @Resource(mappedName = "jms/ConnectionFactory")
          private ConnectionFactory connectionFactory;
      //    @Resource(name = "jms/PerformanceLoggingQueue")
          private Queue queue;
      
          public PerformanceLoggingProducer() {
              try {
                  connectionFactory = (ConnectionFactory) new InitialContext().lookup("jms/ConnectionFactory");
                  queue = (Queue) new InitialContext().lookup("jms/PerformanceLoggingQueue");
              } catch (NamingException ex) {
                  logger.log(Level.SEVERE, null, ex);
              }
          }
      
          public void log(final PerformanceLogTracker tracker) {
      
             if (tracker != null) {
      
              try (Connection connection = connectionFactory.createConnection()) {
      			Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
                  MessageProducer producer = session.createProducer(queue);
                  ObjectMessage message = session.createObjectMessage(tracker);
                  producer.send(message);
               } catch (Exception ex) {
                      logger.log(Level.SEVERE, "An error occurred when logging message: " + tracker, ex);
               }
        
      
             }
          }
      
      }
      

      Thread dump:

      Notice this:

      "main" prio=10 tid=0x00007f5d5c00d000 nid=0xb04 in Object.wait() [0x00007f5d63f61000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000755814468> (a com.sun.jts.CosTransactions.EventSemaphore)
      	at java.lang.Object.wait(Object.java:503)
      	at com.sun.jts.CosTransactions.EventSemaphore.waitEvent(EventSemaphore.java:133)
      	- locked <0x0000000755814468> (a com.sun.jts.CosTransactions.EventSemaphore)
      	at com.sun.jts.CosTransactions.RecoveryManager.waitForResync(RecoveryManager.java:1455)
      	at com.sun.jts.CosTransactions.TransactionFactoryImpl.localCreate(TransactionFactoryImpl.java:180)
      
      2014-02-12 11:04:05
      Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.45-b08 mixed mode):
      
      "__ejb-thread-pool16" daemon prio=10 tid=0x00007f5d08302000 nid=0xc18 waiting on condition [0x00007f5d045ae000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool15" daemon prio=10 tid=0x00007f5d08300000 nid=0xc17 waiting on condition [0x00007f5d046af000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool14" daemon prio=10 tid=0x00007f5d082fe000 nid=0xc16 waiting on condition [0x00007f5d047b0000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool13" daemon prio=10 tid=0x00007f5d08063000 nid=0xc15 waiting on condition [0x00007f5d048b1000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool12" daemon prio=10 tid=0x00007f5d08061000 nid=0xc14 waiting on condition [0x00007f5d049b2000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool11" daemon prio=10 tid=0x00007f5d0805e800 nid=0xc13 waiting on condition [0x00007f5d04ab3000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool10" daemon prio=10 tid=0x00007f5d0805c800 nid=0xc12 waiting on condition [0x00007f5d04bb4000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool9" daemon prio=10 tid=0x00007f5d08752800 nid=0xc11 waiting on condition [0x00007f5d04cb5000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool8" daemon prio=10 tid=0x00007f5d08447000 nid=0xc10 waiting on condition [0x00007f5d04db6000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool7" daemon prio=10 tid=0x00007f5d08446000 nid=0xc0f waiting on condition [0x00007f5d04eb7000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool6" daemon prio=10 tid=0x00007f5d08496000 nid=0xc0e waiting on condition [0x00007f5d04fb8000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool5" daemon prio=10 tid=0x00007f5d0865a800 nid=0xc0d waiting on condition [0x00007f5d052bb000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool4" daemon prio=10 tid=0x00007f5d08659000 nid=0xc0c waiting on condition [0x00007f5d050b9000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool3" daemon prio=10 tid=0x00007f5d08657800 nid=0xc0b waiting on condition [0x00007f5d383c4000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool2" daemon prio=10 tid=0x00007f5d083d5800 nid=0xc0a waiting on condition [0x00007f5d385c6000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "__ejb-thread-pool1" daemon prio=10 tid=0x00007f5d083d4800 nid=0xc09 waiting on condition [0x00007f5d15e6a000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755f3e2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Attach Listener" daemon prio=10 tid=0x00007f5d10022000 nid=0xc03 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "GMS-isConnected-Group-myproject-thread-3" daemon prio=10 tid=0x00007f5d541ee000 nid=0xbbc waiting on condition [0x00007f5d15c68000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000767174f80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-isConnected-Group-myproject-thread-2" daemon prio=10 tid=0x00007f5d541ed000 nid=0xbbb waiting on condition [0x00007f5d384c5000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000767174f80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-isConnected-Group-myproject-thread-1" daemon prio=10 tid=0x00007f5d541ec000 nid=0xbba waiting on condition [0x00007f5d386c7000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000767174f80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Thread-52" daemon prio=10 tid=0x00007f5d3001d800 nid=0xbad runnable [0x00007f5d051ba000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.SocketInputStream.socketRead0(Native Method)
      	at java.net.SocketInputStream.read(SocketInputStream.java:152)
      	at java.net.SocketInputStream.read(SocketInputStream.java:122)
      	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
      	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
      	at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
      	- locked <0x000000075b302158> (a java.io.BufferedInputStream)
      	at com.sun.jndi.ldap.Connection.run(Connection.java:853)
      	at java.lang.Thread.run(Thread.java:744)
      
      "JTS Resync Thread" daemon prio=10 tid=0x00007f5d5e5d8800 nid=0xbab in Object.wait() [0x00007f5d053bc000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000755814448> (a com.sun.jts.CosTransactions.EventSemaphore)
      	at java.lang.Object.wait(Object.java:503)
      	at com.sun.jts.CosTransactions.EventSemaphore.waitEvent(EventSemaphore.java:133)
      	- locked <0x0000000755814448> (a com.sun.jts.CosTransactions.EventSemaphore)
      	at com.sun.jts.CosTransactions.RecoveryManager.proceedWithXARecovery(RecoveryManager.java:948)
      	at com.sun.jts.CosTransactions.RecoveryManager.recover(RecoveryManager.java:492)
      	at com.sun.jts.CosTransactions.ResyncThread.run(RecoveryManager.java:1881)
      
      "Recovery Helper Thread" daemon prio=10 tid=0x00007f5d5e5d7800 nid=0xbaa in Object.wait() [0x00007f5d054bd000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x000000075184dd10> (a org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:134)
      	- locked <0x000000075184dd10> (a org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext)
      	at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2204)
      	at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:579)
      	at org.jvnet.hk2.internal.IterableProviderImpl.get(IterableProviderImpl.java:87)
      	at com.sun.enterprise.resource.recovery.ConnectorsRecoveryResourceHandler.loadXAResourcesAndItsConnections(ConnectorsRecoveryResourceHandler.java:220)
      	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.getAllRecoverableResources(ResourceRecoveryManagerImpl.java:212)
      	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:240)
      	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:337)
      	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.postConstruct(ResourceRecoveryManagerImpl.java:108)
      	at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:281)
      	at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:328)
      	at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:448)
      	at org.jvnet.hk2.internal.SingletonContext.findOrCreate(SingletonContext.java:118)
      	at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2204)
      	at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:579)
      	at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:566)
      	at com.sun.jts.jta.TransactionServiceProperties$RecoveryHelperThread.run(TransactionServiceProperties.java:361)
      
      "iMQReadChannel-3" daemon prio=10 tid=0x00007f5d5e74b800 nid=0xba9 runnable [0x00007f5d055be000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.SocketInputStream.socketRead0(Native Method)
      	at java.net.SocketInputStream.read(SocketInputStream.java:152)
      	at java.net.SocketInputStream.read(SocketInputStream.java:122)
      	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
      	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
      	at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
      	- locked <0x000000075b302b18> (a java.io.BufferedInputStream)
      	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1836)
      	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1804)
      	at com.sun.messaging.jmq.io.Packet.readPacket(Packet.java:1731)
      	- locked <0x000000075b302b40> (a com.sun.messaging.jmq.io.ReadWritePacket)
      	at com.sun.messaging.jmq.io.ReadOnlyPacket.readPacket(ReadOnlyPacket.java:115)
      	at com.sun.messaging.jmq.io.ReadWritePacket.readPacket(ReadWritePacket.java:70)
      	- locked <0x000000075b302b40> (a com.sun.messaging.jmq.io.ReadWritePacket)
      	at com.sun.messaging.jmq.jmsclient.protocol.SocketConnectionHandler.readPacket(SocketConnectionHandler.java:72)
      	at com.sun.messaging.jmq.jmsclient.ProtocolHandler.readPacket(ProtocolHandler.java:1830)
      	at com.sun.messaging.jmq.jmsclient.ReadChannel.run(ReadChannel.java:1275)
      	at java.lang.Thread.run(Thread.java:744)
      
      "imqConnectionFlowControl-3" daemon prio=10 tid=0x00007f5d5e74a000 nid=0xba7 in Object.wait() [0x00007f5d056bf000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x000000075b302de0> (a com.sun.messaging.jmq.jmsclient.FlowControl)
      	at com.sun.messaging.jmq.jmsclient.FlowControl.run(FlowControl.java:307)
      	- locked <0x000000075b302de0> (a com.sun.messaging.jmq.jmsclient.FlowControl)
      	at java.lang.Thread.run(Thread.java:744)
      
      "ContainerBackgroundProcessor[StandardEngine[glassfish-web].StandardHost[server].StandardContext[/ejb-timer-service-app]]" daemon prio=10 tid=0x00007f5d5e495800 nid=0xba5 waiting on condition [0x00007f5d05882000]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
      	at java.lang.Thread.sleep(Native Method)
      	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1801)
      	at java.lang.Thread.run(Thread.java:744)
      
      "imqConsumerReader-2-563587832198206464-1" daemon prio=10 tid=0x00007f5d5e7be800 nid=0xba4 in Object.wait() [0x00007f5d05983000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x000000075d9000e0> (a com.sun.messaging.jmq.jmsclient.SessionQueue)
      	at com.sun.messaging.jmq.jmsclient.SessionQueue.dequeueWait(SessionQueue.java:257)
      	- locked <0x000000075d9000e0> (a com.sun.messaging.jmq.jmsclient.SessionQueue)
      	at com.sun.messaging.jmq.jmsclient.ConsumerReader.run(ConsumerReader.java:168)
      	at java.lang.Thread.run(Thread.java:744)
      
      "iMQReadChannel-2" daemon prio=10 tid=0x00007f5d5e1c8800 nid=0xba3 runnable [0x00007f5d15f6b000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.SocketInputStream.socketRead0(Native Method)
      	at java.net.SocketInputStream.read(SocketInputStream.java:152)
      	at java.net.SocketInputStream.read(SocketInputStream.java:122)
      	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
      	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
      	at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
      	- locked <0x000000075d900a50> (a java.io.BufferedInputStream)
      	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1836)
      	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1804)
      	at com.sun.messaging.jmq.io.Packet.readPacket(Packet.java:1731)
      	- locked <0x000000075d900a78> (a com.sun.messaging.jmq.io.ReadWritePacket)
      	at com.sun.messaging.jmq.io.ReadOnlyPacket.readPacket(ReadOnlyPacket.java:115)
      	at com.sun.messaging.jmq.io.ReadWritePacket.readPacket(ReadWritePacket.java:70)
      	- locked <0x000000075d900a78> (a com.sun.messaging.jmq.io.ReadWritePacket)
      	at com.sun.messaging.jmq.jmsclient.protocol.SocketConnectionHandler.readPacket(SocketConnectionHandler.java:72)
      	at com.sun.messaging.jmq.jmsclient.ProtocolHandler.readPacket(ProtocolHandler.java:1830)
      	at com.sun.messaging.jmq.jmsclient.ReadChannel.run(ReadChannel.java:1275)
      	at java.lang.Thread.run(Thread.java:744)
      
      "imqConnectionFlowControl-2" daemon prio=10 tid=0x00007f5d5e1c7800 nid=0xba2 in Object.wait() [0x00007f5d15d69000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x000000075d900d18> (a com.sun.messaging.jmq.jmsclient.FlowControl)
      	at com.sun.messaging.jmq.jmsclient.FlowControl.run(FlowControl.java:307)
      	- locked <0x000000075d900d18> (a com.sun.messaging.jmq.jmsclient.FlowControl)
      	at java.lang.Thread.run(Thread.java:744)
      
      "ContainerBackgroundProcessor[StandardEngine[glassfish-web].StandardHost[server].StandardContext[/smb-access-svc]]" daemon prio=10 tid=0x00007f5d5e16c800 nid=0xb9b waiting on condition [0x00007f5d05ad9000]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
      	at java.lang.Thread.sleep(Native Method)
      	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1801)
      	at java.lang.Thread.run(Thread.java:744)
      
      "imqConsumerReader-1-563587832187488512-0" daemon prio=10 tid=0x00007f5d5d645000 nid=0xb9a in Object.wait() [0x00007f5d05bda000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007675442c0> (a com.sun.messaging.jmq.jmsclient.SessionQueue)
      	at com.sun.messaging.jmq.jmsclient.SessionQueue.dequeueWait(SessionQueue.java:257)
      	- locked <0x00000007675442c0> (a com.sun.messaging.jmq.jmsclient.SessionQueue)
      	at com.sun.messaging.jmq.jmsclient.ConsumerReader.run(ConsumerReader.java:168)
      	at java.lang.Thread.run(Thread.java:744)
      
      "iMQReadChannel-1" daemon prio=10 tid=0x00007f5d5db24000 nid=0xb99 runnable [0x00007f5d05cdb000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.SocketInputStream.socketRead0(Native Method)
      	at java.net.SocketInputStream.read(SocketInputStream.java:152)
      	at java.net.SocketInputStream.read(SocketInputStream.java:122)
      	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
      	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
      	at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
      	- locked <0x0000000767544bd8> (a java.io.BufferedInputStream)
      	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1836)
      	at com.sun.messaging.jmq.io.Packet.readFully(Packet.java:1804)
      	at com.sun.messaging.jmq.io.Packet.readPacket(Packet.java:1731)
      	- locked <0x0000000767544c58> (a com.sun.messaging.jmq.io.ReadWritePacket)
      	at com.sun.messaging.jmq.io.ReadOnlyPacket.readPacket(ReadOnlyPacket.java:115)
      	at com.sun.messaging.jmq.io.ReadWritePacket.readPacket(ReadWritePacket.java:70)
      	- locked <0x0000000767544c58> (a com.sun.messaging.jmq.io.ReadWritePacket)
      	at com.sun.messaging.jmq.jmsclient.protocol.SocketConnectionHandler.readPacket(SocketConnectionHandler.java:72)
      	at com.sun.messaging.jmq.jmsclient.ProtocolHandler.readPacket(ProtocolHandler.java:1830)
      	at com.sun.messaging.jmq.jmsclient.ReadChannel.run(ReadChannel.java:1275)
      	at java.lang.Thread.run(Thread.java:744)
      
      "imqConnectionFlowControl-1" daemon prio=10 tid=0x00007f5d5db1e800 nid=0xb98 in Object.wait() [0x00007f5d05ddc000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000767544ef8> (a com.sun.messaging.jmq.jmsclient.FlowControl)
      	at com.sun.messaging.jmq.jmsclient.FlowControl.run(FlowControl.java:307)
      	- locked <0x0000000767544ef8> (a com.sun.messaging.jmq.jmsclient.FlowControl)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-DistributedStateCache-Group-myproject-thread-1" daemon prio=10 tid=0x00007f5d0d867800 nid=0xb97 waiting on condition [0x00007f5d05edd000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f68b90> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-validateMasterChangeEvents-Group-myproject-thread-1" daemon prio=10 tid=0x00007f5d1c01d000 nid=0xb96 waiting on condition [0x00007f5d05fde000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f68c40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-processNotify-Group-myproject-thread-2" daemon prio=10 tid=0x000000000078a000 nid=0xb95 waiting on condition [0x00007f5d060df000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65c98> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-processNotify-Group-myproject-thread-1" daemon prio=10 tid=0x0000000000789000 nid=0xb94 waiting on condition [0x00007f5d071f0000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65c98> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-McastMsgProcessor-Group-myproject-thread-10" daemon prio=10 tid=0x00007f5d2c042000 nid=0xb91 waiting on condition [0x00007f5d061e0000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Timer-3" daemon prio=10 tid=0x00007f5d48016000 nid=0xb90 in Object.wait() [0x00007f5d062e1000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007675452e8> (a java.util.TaskQueue)
      	at java.util.TimerThread.mainLoop(Timer.java:552)
      	- locked <0x00000007675452e8> (a java.util.TaskQueue)
      	at java.util.TimerThread.run(Timer.java:505)
      
      "GMS-McastMsgProcessor-Group-myproject-thread-9" daemon prio=10 tid=0x00007f5d2c040000 nid=0xb8f waiting on condition [0x00007f5d063e2000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-McastMsgProcessor-Group-myproject-thread-8" daemon prio=10 tid=0x00007f5d2c03d800 nid=0xb8e waiting on condition [0x00007f5d064e3000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Thread-27" prio=10 tid=0x00007f5d5d89e800 nid=0xb8d in Object.wait() [0x00007f5d065e4000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000767545468> (a com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive)
      	at java.lang.Object.wait(Object.java:503)
      	at com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive.run(Util.java:818)
      	- locked <0x0000000767545468> (a com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive)
      
      "GMS-McastMsgProcessor-Group-myproject-thread-7" daemon prio=10 tid=0x00007f5d2c03b800 nid=0xb8c waiting on condition [0x00007f5d066e5000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "SelectorThread" daemon prio=10 tid=0x00007f5d5da26000 nid=0xb8b runnable [0x00007f5d067e6000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x0000000767545690> (a sun.nio.ch.Util$2)
      	- locked <0x00000007675456a0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x0000000767545648> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at com.sun.corba.ee.impl.transport.SelectorImpl.run(SelectorImpl.java:303)
      
      "GMS-McastMsgProcessor-Group-myproject-thread-6" daemon prio=10 tid=0x00007f5d2c039800 nid=0xb8a waiting on condition [0x00007f5d068e7000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-McastMsgProcessor-Group-myproject-thread-5" daemon prio=10 tid=0x00007f5d2c038000 nid=0xb89 waiting on condition [0x00007f5d069e8000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-McastMsgProcessor-Group-myproject-thread-4" daemon prio=10 tid=0x00007f5d2c02b800 nid=0xb88 waiting on condition [0x00007f5d06ae9000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-McastMsgProcessor-Group-myproject-thread-3" daemon prio=10 tid=0x00007f5d2c029800 nid=0xb87 waiting on condition [0x00007f5d06bea000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-McastMsgProcessor-Group-myproject-thread-2" daemon prio=10 tid=0x00007f5d2c027800 nid=0xb86 waiting on condition [0x00007f5d06ceb000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-McastMsgProcessor-Group-myproject-thread-1" daemon prio=10 tid=0x00007f5d2c04b000 nid=0xb85 waiting on condition [0x00007f5d06dec000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f65d48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS FailureVerifier Thread for Group-myproject" daemon prio=10 tid=0x00007f5d5cd84800 nid=0xb84 in Object.wait() [0x00007f5d06eed000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f68e88> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at com.sun.enterprise.mgmt.HealthMonitor$FailureVerifier.run(HealthMonitor.java:1430)
      	- locked <0x0000000752f68e88> (a java.lang.Object)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS InDoubtPeerDetector Thread for Group-myproject" daemon prio=10 tid=0x00007f5d5cd82000 nid=0xb83 in Object.wait() [0x00007f5d06fee000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f68f18> (a java.lang.Object)
      	at com.sun.enterprise.mgmt.HealthMonitor$InDoubtPeerDetector.run(HealthMonitor.java:1290)
      	- locked <0x0000000752f68f18> (a java.lang.Object)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS HealthMonitor for Group-myproject" daemon prio=10 tid=0x00007f5d5cd80000 nid=0xb82 in Object.wait() [0x00007f5d070ef000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f68fa8> (a java.lang.Object)
      	at com.sun.enterprise.mgmt.HealthMonitor.run(HealthMonitor.java:817)
      	- locked <0x0000000752f68fa8> (a java.lang.Object)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS MasterNode processOutstandingChangeEvents Group-myproject" daemon prio=10 tid=0x00007f5d5d247800 nid=0xb80 in Object.wait() [0x00007f5d072f1000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f690b0> (a java.util.TreeSet)
      	at java.lang.Object.wait(Object.java:503)
      	at com.sun.enterprise.mgmt.MasterNode$ProcessOutstandingMessagesTask.run(MasterNode.java:1859)
      	- locked <0x0000000752f690b0> (a java.util.TreeSet)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Timer-2" daemon prio=10 tid=0x00007f5d5d23e800 nid=0xb7f in Object.wait() [0x00007f5d073f2000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f69140> (a java.util.TaskQueue)
      	at java.lang.Object.wait(Object.java:503)
      	at java.util.TimerThread.mainLoop(Timer.java:526)
      	- locked <0x0000000752f69140> (a java.util.TaskQueue)
      	at java.util.TimerThread.run(Timer.java:505)
      
      "IP Multicast Listener for /228.9.219.87:9336" daemon prio=10 tid=0x00007f5d5d279000 nid=0xb7e runnable [0x00007f5d074f3000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.PlainDatagramSocketImpl.receive0(Native Method)
      	- locked <0x0000000752f691d8> (a java.net.PlainDatagramSocketImpl)
      	at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145)
      	- locked <0x0000000752f691d8> (a java.net.PlainDatagramSocketImpl)
      	at java.net.DatagramSocket.receive(DatagramSocket.java:786)
      	- locked <0x00000007694c5448> (a java.net.DatagramPacket)
      	- locked <0x0000000752f69208> (a java.net.MulticastSocket)
      	at com.sun.enterprise.mgmt.transport.BlockingIOMulticastSender.run(BlockingIOMulticastSender.java:254)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(20)" daemon prio=10 tid=0x00007f5d5d259800 nid=0xb7d in Object.wait() [0x00007f5d075f4000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(19)" daemon prio=10 tid=0x00007f5d5d257800 nid=0xb7c in Object.wait() [0x00007f5d076f5000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(18)" daemon prio=10 tid=0x00007f5d5d255800 nid=0xb7b in Object.wait() [0x00007f5d077f6000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(17)" daemon prio=10 tid=0x00007f5d5d2be800 nid=0xb7a in Object.wait() [0x00007f5d078f7000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(16)" daemon prio=10 tid=0x00007f5d5d2bc800 nid=0xb79 in Object.wait() [0x00007f5d079f8000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(15)" daemon prio=10 tid=0x00007f5d5d2ba800 nid=0xb78 in Object.wait() [0x00007f5d07af9000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(14)" daemon prio=10 tid=0x00007f5d5d2b8800 nid=0xb77 in Object.wait() [0x00007f5d07bfa000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(13)" daemon prio=10 tid=0x00007f5d5d2b6800 nid=0xb76 in Object.wait() [0x00007f5d07cfb000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(12)" daemon prio=10 tid=0x00007f5d5d2b4800 nid=0xb75 in Object.wait() [0x00007f5d07dfc000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(11)" daemon prio=10 tid=0x00007f5d5d2b2000 nid=0xb74 in Object.wait() [0x00007f5d07efd000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(10)" daemon prio=10 tid=0x00007f5d5d2b0000 nid=0xb73 in Object.wait() [0x00007f5d07ffe000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(9)" daemon prio=10 tid=0x00007f5d5d2ae000 nid=0xb72 in Object.wait() [0x00007f5d141d8000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(8)" daemon prio=10 tid=0x00007f5d5d2ac000 nid=0xb71 in Object.wait() [0x00007f5d142d9000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(7)" daemon prio=10 tid=0x00007f5d5d2aa000 nid=0xb70 in Object.wait() [0x00007f5d143da000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(6)" daemon prio=10 tid=0x00007f5d5d2a7800 nid=0xb6f in Object.wait() [0x00007f5d144db000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(5)" daemon prio=10 tid=0x00007f5d5d2a5800 nid=0xb6e in Object.wait() [0x00007f5d145dc000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(4)" daemon prio=10 tid=0x00007f5d5d2a3800 nid=0xb6d in Object.wait() [0x00007f5d146dd000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(3)" daemon prio=10 tid=0x00007f5d5d2a1800 nid=0xb6c in Object.wait() [0x00007f5d147de000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(2)" daemon prio=10 tid=0x00007f5d5d29f800 nid=0xb6b in Object.wait() [0x00007f5d148df000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS-GrizzlyControllerThreadPool-Group-myproject(1)" daemon prio=10 tid=0x00007f5d5d75a000 nid=0xb6a in Object.wait() [0x00007f5d149e0000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752f65f90> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000752f65f90> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Grizzly-kernel(2) SelectorRunner" daemon prio=10 tid=0x00007f5d5d758800 nid=0xb69 runnable [0x00007f5d14ae1000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x0000000752f66618> (a sun.nio.ch.Util$2)
      	- locked <0x0000000752f66628> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x0000000752f665d0> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112)
      	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:333)
      	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:274)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Grizzly-kernel(1) SelectorRunner" daemon prio=10 tid=0x00007f5d5d757000 nid=0xb68 runnable [0x00007f5d14be2000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x0000000752f667e0> (a sun.nio.ch.Util$2)
      	- locked <0x0000000752f667f0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x0000000752f66798> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112)
      	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:333)
      	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:274)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS ViewWindowThread Group-myproject" daemon prio=10 tid=0x00007f5d5d744800 nid=0xb67 waiting on condition [0x00007f5d14ce3000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f668e8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at com.sun.enterprise.ee.cms.impl.base.ViewWindowImpl.run(ViewWindowImpl.java:189)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS MessageWindowThread Group-myproject" daemon prio=10 tid=0x00007f5d5d747000 nid=0xb66 waiting on condition [0x00007f5d14de4000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f669d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at com.sun.enterprise.ee.cms.impl.base.MessageWindow.run(MessageWindow.java:108)
      	at java.lang.Thread.run(Thread.java:744)
      
      "GMS SignalHandler for Group-myproject thread" daemon prio=10 tid=0x00007f5d5d2e1000 nid=0xb65 waiting on condition [0x00007f5d14ee5000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000752f69a40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at com.sun.enterprise.ee.cms.impl.common.SignalHandler.run(SignalHandler.java:90)
      	at java.lang.Thread.run(Thread.java:744)
      
      "process reaper" daemon prio=10 tid=0x00007f5d1000a000 nid=0xb62 runnable [0x00007f5d14f1e000]
         java.lang.Thread.State: RUNNABLE
      	at java.lang.UNIXProcess.waitForProcessExit(Native Method)
      	at java.lang.UNIXProcess.access$200(UNIXProcess.java:54)
      	at java.lang.UNIXProcess$3.run(UNIXProcess.java:174)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "process reaper" daemon prio=10 tid=0x00007f5d10008800 nid=0xb5e runnable [0x00007f5d14f57000]
         java.lang.Thread.State: RUNNABLE
      	at java.lang.UNIXProcess.waitForProcessExit(Native Method)
      	at java.lang.UNIXProcess.access$200(UNIXProcess.java:54)
      	at java.lang.UNIXProcess$3.run(UNIXProcess.java:174)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "process reaper" daemon prio=10 tid=0x00007f5d10007000 nid=0xb5a runnable [0x00007f5d3806f000]
         java.lang.Thread.State: RUNNABLE
      	at java.lang.UNIXProcess.waitForProcessExit(Native Method)
      	at java.lang.UNIXProcess.access$200(UNIXProcess.java:54)
      	at java.lang.UNIXProcess$3.run(UNIXProcess.java:174)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "process reaper" daemon prio=10 tid=0x00007f5d10005800 nid=0xb56 runnable [0x00007f5d380a8000]
         java.lang.Thread.State: RUNNABLE
      	at java.lang.UNIXProcess.waitForProcessExit(Native Method)
      	at java.lang.UNIXProcess.access$200(UNIXProcess.java:54)
      	at java.lang.UNIXProcess$3.run(UNIXProcess.java:174)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "connector-timer-proxy" daemon prio=10 tid=0x00007f5d5d80f800 nid=0xb54 in Object.wait() [0x00007f5d15058000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007563003f8> (a java.util.TaskQueue)
      	at java.util.TimerThread.mainLoop(Timer.java:552)
      	- locked <0x00000007563003f8> (a java.util.TaskQueue)
      	at java.util.TimerThread.run(Timer.java:505)
      
      "Timer-1" daemon prio=10 tid=0x00007f5d5d814000 nid=0xb53 in Object.wait() [0x00007f5d15159000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007563004c8> (a java.util.TaskQueue)
      	at java.lang.Object.wait(Object.java:503)
      	at java.util.TimerThread.mainLoop(Timer.java:526)
      	- locked <0x00000007563004c8> (a java.util.TaskQueue)
      	at java.util.TimerThread.run(Timer.java:505)
      
      "p: thread-pool-1; w: 5" daemon prio=10 tid=0x00007f5d5d4ea000 nid=0xb52 in Object.wait() [0x00007f5d15460000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
      	at com.sun.corba.ee.impl.threadpool.WorkQueueImpl.requestWork(WorkQueueImpl.java:124)
      	- locked <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
      	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:518)
      
      "p: thread-pool-1; w: 4" daemon prio=10 tid=0x00007f5d5d4e8000 nid=0xb51 in Object.wait() [0x00007f5d15561000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
      	at com.sun.corba.ee.impl.threadpool.WorkQueueImpl.requestWork(WorkQueueImpl.java:124)
      	- locked <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
      	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:518)
      
      "p: thread-pool-1; w: 3" daemon prio=10 tid=0x00007f5d5d4e6000 nid=0xb50 in Object.wait() [0x00007f5d15662000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
      	at com.sun.corba.ee.impl.threadpool.WorkQueueImpl.requestWork(WorkQueueImpl.java:124)
      	- locked <0x000000075440c200> (a com.sun.corba.ee.impl.threadpool.WorkQueueImpl)
      	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:518)
      
      "p: thread-pool-1; w: 2" daemon prio=10 tid=0x00007f5d5d4e3800 nid=0xb4f runnable [0x00007f5d15763000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.PlainSocketImpl.socketAccept(Native Method)
      	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
      	at java.net.ServerSocket.implAccept(ServerSocket.java:530)
      	at sun.security.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:317)
      	at com.sun.corba.ee.impl.transport.AcceptorImpl.getAcceptedSocket(AcceptorImpl.java:148)
      	at com.sun.corba.ee.impl.transport.ListenerThreadImpl.doWork(ListenerThreadImpl.java:114)
      	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497)
      	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540)
      
      "p: thread-pool-1; w: 1" daemon prio=10 tid=0x00007f5d5d4ed000 nid=0xb4e runnable [0x00007f5d15864000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.PlainSocketImpl.socketAccept(Native Method)
      	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
      	at java.net.ServerSocket.implAccept(ServerSocket.java:530)
      	at sun.security.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:317)
      	at com.sun.corba.ee.impl.transport.AcceptorImpl.getAcceptedSocket(AcceptorImpl.java:148)
      	at com.sun.corba.ee.impl.transport.ListenerThreadImpl.doWork(ListenerThreadImpl.java:114)
      	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497)
      	at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540)
      
      "dol-jar-scanner" daemon prio=10 tid=0x00007f5d5d0d0800 nid=0xb4d waiting on condition [0x00007f5d15965000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x000000075276f448> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "dol-jar-scanner" daemon prio=10 tid=0x00007f5d5d0c7800 nid=0xb4c waiting on condition [0x00007f5d15a66000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x000000075276f448> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Timer-0" daemon prio=10 tid=0x00007f5d5d28a800 nid=0xb4b in Object.wait() [0x00007f5d15b67000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000755f002e8> (a java.util.TaskQueue)
      	at java.util.TimerThread.mainLoop(Timer.java:552)
      	- locked <0x0000000755f002e8> (a java.util.TaskQueue)
      	at java.util.TimerThread.run(Timer.java:505)
      
      "pool-11-thread-1" daemon prio=10 tid=0x00007f5d5d376000 nid=0xb46 waiting on condition [0x00007f5d1606c000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000007522dfeb8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:162)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "pool-10-thread-1" daemon prio=10 tid=0x00007f5d5d374000 nid=0xb45 waiting on condition [0x00007f5d1616d000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000007518ea718> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:162)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "pool-9-thread-1" daemon prio=10 tid=0x00007f5d5cd01000 nid=0xb44 waiting on condition [0x00007f5d381c2000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000007522df948> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:162)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "pool-8-thread-1" daemon prio=10 tid=0x00007f5d5c877000 nid=0xb43 waiting on condition [0x00007f5d382c3000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000007522dfb48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:162)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "ContainerBackgroundProcessor[StandardEngine[glassfish-web].StandardHost[server].StandardContext[/highcharts-export-web]]" daemon prio=10 tid=0x00007f5d5d1fd800 nid=0xb3d waiting on condition [0x00007f5d388df000]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
      	at java.lang.Thread.sleep(Native Method)
      	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1801)
      	at java.lang.Thread.run(Thread.java:744)
      
      "pool-2-thread-1" prio=10 tid=0x00007f5d5d1fc000 nid=0xb3c waiting on condition [0x00007f5d389e0000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000753b00068> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "ContainerBackgroundProcessor[StandardEngine[glassfish-web]]" daemon prio=10 tid=0x00007f5d5cee6800 nid=0xb3b waiting on condition [0x00007f5d38ae1000]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
      	at java.lang.Thread.sleep(Native Method)
      	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1801)
      	at java.lang.Thread.run(Thread.java:744)
      
      "ContainerBackgroundProcessor[StandardEngine[glassfish-web].StandardHost[server].StandardContext[]]" daemon prio=10 tid=0x00007f5d5ceb4800 nid=0xb3a waiting on condition [0x00007f5d38be2000]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
      	at java.lang.Thread.sleep(Native Method)
      	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1801)
      	at java.lang.Thread.run(Thread.java:744)
      
      "deployment-jar-scanner" daemon prio=10 tid=0x00007f5d5c937800 nid=0xb39 waiting on condition [0x00007f5d39487000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000007558ce390> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "deployment-jar-scanner" daemon prio=10 tid=0x00007f5d5c92e000 nid=0xb38 waiting on condition [0x00007f5d39588000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000007558ce390> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "transaction-manager" daemon prio=10 tid=0x00007f5d5c899000 nid=0xb37 in Object.wait() [0x00007f5d39689000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000755500308> (a java.util.TaskQueue)
      	at java.lang.Object.wait(Object.java:503)
      	at java.util.TimerThread.mainLoop(Timer.java:526)
      	- locked <0x0000000755500308> (a java.util.TaskQueue)
      	at java.util.TimerThread.run(Timer.java:505)
      
      "admin-listener-kernel(1) SelectorRunner" daemon prio=10 tid=0x00007f5d5c7c9800 nid=0xb36 runnable [0x00007f5d3978a000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000007555012d0> (a sun.nio.ch.Util$2)
      	- locked <0x00000007555012e0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x0000000755501288> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112)
      	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:333)
      	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:274)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "admin-listener-0" daemon prio=10 tid=0x00007f5d5c7c8000 nid=0xb35 in Object.wait() [0x00007f5d3988b000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555003a0> (a java.lang.Object)
      	at org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:172)
      	- locked <0x00000007555003a0> (a java.lang.Object)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "admin-listener(5)" daemon prio=10 tid=0x00007f5d5c7c5800 nid=0xb34 in Object.wait() [0x00007f5d3998c000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555004b8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x00000007555004b8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "admin-listener(4)" daemon prio=10 tid=0x00007f5d5c7c3800 nid=0xb33 in Object.wait() [0x00007f5d39a8d000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555004b8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x00000007555004b8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "admin-listener(3)" daemon prio=10 tid=0x00007f5d5c7c1800 nid=0xb32 in Object.wait() [0x00007f5d39b8e000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555004b8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x00000007555004b8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "admin-listener(2)" daemon prio=10 tid=0x00007f5d5c7bc800 nid=0xb31 in Object.wait() [0x00007f5d39c8f000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555004b8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x00000007555004b8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "admin-listener(1)" daemon prio=10 tid=0x00007f5d5c7ba800 nid=0xb30 in Object.wait() [0x00007f5d39d90000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555004b8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x00000007555004b8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-2-kernel(1) SelectorRunner" daemon prio=10 tid=0x00007f5d5c79e800 nid=0xb2f runnable [0x00007f5d39e91000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x0000000755501600> (a sun.nio.ch.Util$2)
      	- locked <0x0000000755501610> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000007555015b8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112)
      	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:333)
      	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:274)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-2-0" daemon prio=10 tid=0x00007f5d5c79d000 nid=0xb2e in Object.wait() [0x00007f5d39f92000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555006c0> (a java.lang.Object)
      	at org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:172)
      	- locked <0x00000007555006c0> (a java.lang.Object)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-2(5)" daemon prio=10 tid=0x00007f5d5c79a000 nid=0xb2d in Object.wait() [0x00007f5d3a093000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555007d8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x00000007555007d8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-2(4)" daemon prio=10 tid=0x00007f5d5c798000 nid=0xb2c in Object.wait() [0x00007f5d3a194000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555007d8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x00000007555007d8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-2(3)" daemon prio=10 tid=0x00007f5d5c796000 nid=0xb2b in Object.wait() [0x00007f5d3a295000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555007d8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x00000007555007d8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-2(2)" daemon prio=10 tid=0x00007f5d5c794000 nid=0xb2a in Object.wait() [0x00007f5d5815b000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555007d8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x00000007555007d8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-2(1)" daemon prio=10 tid=0x00007f5d5c782000 nid=0xb29 in Object.wait() [0x00007f5d5825c000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555007d8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x00000007555007d8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-1-kernel(1) SelectorRunner" daemon prio=10 tid=0x00007f5d5c75e800 nid=0xb28 runnable [0x00007f5d5835d000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x0000000755501918> (a sun.nio.ch.Util$2)
      	- locked <0x0000000755501928> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000007555018d0> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112)
      	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:333)
      	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:274)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-1-0" daemon prio=10 tid=0x00007f5d5c757800 nid=0xb27 in Object.wait() [0x00007f5d5845e000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007555009e0> (a java.lang.Object)
      	at org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:172)
      	- locked <0x00000007555009e0> (a java.lang.Object)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-1(5)" daemon prio=10 tid=0x00007f5d5c754800 nid=0xb26 in Object.wait() [0x00007f5d5855f000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000755500af8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000755500af8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-1(4)" daemon prio=10 tid=0x00007f5d5c752800 nid=0xb25 in Object.wait() [0x00007f5d58660000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000755500af8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000755500af8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-1(3)" daemon prio=10 tid=0x00007f5d5c750800 nid=0xb24 in Object.wait() [0x00007f5d58761000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000755500af8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000755500af8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-1(2)" daemon prio=10 tid=0x00007f5d5c74e800 nid=0xb23 in Object.wait() [0x00007f5d58862000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000755500af8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000755500af8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "http-listener-1(1)" daemon prio=10 tid=0x00007f5d5c73a800 nid=0xb22 in Object.wait() [0x00007f5d58963000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000755500af8> (a java.lang.Object)
      	at java.lang.Object.wait(Object.java:503)
      	at org.glassfish.grizzly.threadpool.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:198)
      	- locked <0x0000000755500af8> (a java.lang.Object)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:556)
      	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Grizzly-HttpSession-Expirer" daemon prio=10 tid=0x00007f5d5c707000 nid=0xb21 waiting on condition [0x00007f5d58c6c000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000755500c40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Thread-4" daemon prio=10 tid=0x00007f5d5c5d0000 nid=0xb20 waiting on condition [0x00007f5d58d6d000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x000000075184d880> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at com.sun.common.util.logging.LoggingOutputStream.log(LoggingOutputStream.java:146)
      	at com.sun.common.util.logging.LoggingOutputStream$1.run(LoggingOutputStream.java:125)
      
      "Thread-3" daemon prio=10 tid=0x00007f5d5c5bd800 nid=0xb1f waiting on condition [0x00007f5d58e6e000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x000000075184d990> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at com.sun.common.util.logging.LoggingOutputStream.log(LoggingOutputStream.java:146)
      	at com.sun.common.util.logging.LoggingOutputStream$1.run(LoggingOutputStream.java:125)
      
      "Thread-2" daemon prio=10 tid=0x00007f5d5c5ca000 nid=0xb1e waiting on condition [0x00007f5d58f6f000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x000000075184daa0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at com.sun.enterprise.server.logging.GFFileHandler.log(GFFileHandler.java:825)
      	at com.sun.enterprise.server.logging.GFFileHandler$3.run(GFFileHandler.java:540)
      
      "pool-1-thread-1" daemon prio=10 tid=0x00007f5d5c49e800 nid=0xb1d waiting on condition [0x00007f5d591c4000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x000000075184dc00> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      
      "FelixStartLevel" daemon prio=10 tid=0x00007f5d5c2f2800 nid=0xb1c in Object.wait() [0x00007f5d59e66000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x000000075094af88> (a java.util.ArrayList)
      	at java.lang.Object.wait(Object.java:503)
      	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:279)
      	- locked <0x000000075094af88> (a java.util.ArrayList)
      	at java.lang.Thread.run(Thread.java:744)
      
      "FelixDispatchQueue" daemon prio=10 tid=0x00007f5d08152800 nid=0xb18 in Object.wait() [0x00007f5d59d65000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752880f40> (a java.util.ArrayList)
      	at java.lang.Object.wait(Object.java:503)
      	at org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:1063)
      	- locked <0x0000000752880f40> (a java.util.ArrayList)
      	at org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:54)
      	at org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:101)
      	at java.lang.Thread.run(Thread.java:744)
      
      "Service Thread" daemon prio=10 tid=0x00007f5d5c195000 nid=0xb15 runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C2 CompilerThread1" daemon prio=10 tid=0x00007f5d5c192800 nid=0xb14 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C2 CompilerThread0" daemon prio=10 tid=0x00007f5d5c190000 nid=0xb13 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "Signal Dispatcher" daemon prio=10 tid=0x00007f5d5c114800 nid=0xb12 runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f5d5c113000 nid=0xb11 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "Finalizer" daemon prio=10 tid=0x00007f5d5c0f2000 nid=0xb10 in Object.wait() [0x00007f5d5a5e4000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007528814e8> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
      	- locked <0x00000007528814e8> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
      	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
      
      "Reference Handler" daemon prio=10 tid=0x00007f5d5c0ee000 nid=0xb0f in Object.wait() [0x00007f5d5a6e5000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000752881580> (a java.lang.ref.Reference$Lock)
      	at java.lang.Object.wait(Object.java:503)
      	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
      	- locked <0x0000000752881580> (a java.lang.ref.Reference$Lock)
      
      "main" prio=10 tid=0x00007f5d5c00d000 nid=0xb04 in Object.wait() [0x00007f5d63f61000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000755814468> (a com.sun.jts.CosTransactions.EventSemaphore)
      	at java.lang.Object.wait(Object.java:503)
      	at com.sun.jts.CosTransactions.EventSemaphore.waitEvent(EventSemaphore.java:133)
      	- locked <0x0000000755814468> (a com.sun.jts.CosTransactions.EventSemaphore)
      	at com.sun.jts.CosTransactions.RecoveryManager.waitForResync(RecoveryManager.java:1455)
      	at com.sun.jts.CosTransactions.TransactionFactoryImpl.localCreate(TransactionFactoryImpl.java:180)
      	at com.sun.jts.CosTransactions.CurrentImpl.begin(CurrentImpl.java:428)
      	at com.sun.jts.jta.TransactionManagerImpl.begin(TransactionManagerImpl.java:296)
      	at com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate.beginJTS(JavaEETransactionManagerJTSDelegate.java:498)
      	at com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate.startJTSTx(JavaEETransactionManagerJTSDelegate.java:401)
      	at com.sun.enterprise.transaction.JavaEETransactionManagerSimplified.startJTSTx(JavaEETransactionManagerSimplified.java:429)
      	at com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate.enlistLAOResource(JavaEETransactionManagerJTSDelegate.java:318)
      	at com.sun.enterprise.transaction.JavaEETransactionManagerSimplified.enlistResource(JavaEETransactionManagerSimplified.java:356)
      	at com.sun.enterprise.resource.rm.ResourceManagerImpl.registerResource(ResourceManagerImpl.java:152)
      	at com.sun.enterprise.resource.rm.ResourceManagerImpl.enlistResource(ResourceManagerImpl.java:112)
      	at com.sun.enterprise.resource.pool.PoolManagerImpl.getResource(PoolManagerImpl.java:211)
      	at com.sun.enterprise.connectors.ConnectionManagerImpl.getResource(ConnectionManagerImpl.java:354)
      	at com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetConnection(ConnectionManagerImpl.java:307)
      	at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:196)
      	at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:171)
      	at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:166)
      	at com.sun.messaging.jms.ra.ConnectionFactoryAdapter._allocateConnection(ConnectionFactoryAdapter.java:212)
      	at com.sun.messaging.jms.ra.ConnectionFactoryAdapter.createConnection(ConnectionFactoryAdapter.java:170)
      	at com.sun.messaging.jms.ra.ConnectionFactoryAdapter.createConnection(ConnectionFactoryAdapter.java:152)
      	at com.me.myproject.logging.performance.PerformanceLoggingProducer.log(PerformanceLoggingProducer.java:45)
      	at com.me.myproject.logging.performance.PerformanceLoggingProducer$Proxy$_$$_WeldClientProxy.log(Unknown Source)
      	at com.me.myproject.ejbinterface.common.RemoteBeanProducer$1.invoke(RemoteBeanProducer.java:128)
      	at com.sun.proxy.$Proxy446.getProperty(Unknown Source)
      	at com.me.myproject.releasedocuments.basedirectory.BaseDirectoryPropertyInterface.getAppPropertyForBaseDirectory(BaseDirectoryPropertyInterface.java:74)
      	at com.me.myproject.releasedocuments.basedirectory.BaseDirectoryPropertyInterface.getBaseDirectory(BaseDirectoryPropertyInterface.java:42)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
      	at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
      	at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4695)
      	at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:630)
      	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
      	at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:582)
      	at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:46)
      	at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
      	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
      	at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:582)
      	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
      	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
      	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
      	at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
      	at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4667)
      	at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4655)
      	at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
      	at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
      	at com.sun.proxy.$Proxy444.getBaseDirectory(Unknown Source)
      	at com.me.myproject.releasedocuments.basedirectory.__EJB31_Generated__BaseDirectoryPropertyInterface__Intf____Bean__.getBaseDirectory(Unknown Source)
      	at com.me.myproject.releasedocuments.ReleaseDocumentsInit.initialize(ReleaseDocumentsInit.java:37)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at com.sun.ejb.containers.interceptors.BeanCallbackInterceptor.intercept(InterceptorManager.java:1035)
      	at com.sun.ejb.containers.interceptors.CallbackChainImpl.invokeNext(CallbackChainImpl.java:72)
      	at com.sun.ejb.containers.interceptors.CallbackInvocationContext.proceed(CallbackInvocationContext.java:205)
      	at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:55)
      	at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at com.sun.ejb.containers.interceptors.CallbackInterceptor.intercept(InterceptorManager.java:986)
      	at com.sun.ejb.containers.interceptors.CallbackChainImpl.invokeNext(CallbackChainImpl.java:72)
      	at com.sun.ejb.containers.interceptors.CallbackInvocationContext.proceed(CallbackInvocationContext.java:205)
      	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
      	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.init(SystemInterceptorProxy.java:125)
      	at sun.reflect.GeneratedMethodAccessor83.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at com.sun.ejb.containers.interceptors.CallbackInterceptor.intercept(InterceptorManager.java:986)
      	at com.sun.ejb.containers.interceptors.CallbackChainImpl.invokeNext(CallbackChainImpl.java:72)
      	at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:412)
      	at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:375)
      	at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:1949)
      	at com.sun.ejb.containers.AbstractSingletonContainer.createSingletonEJB(AbstractSingletonContainer.java:475)
      	at com.sun.ejb.containers.AbstractSingletonContainer.access$000(AbstractSingletonContainer.java:81)
      	at com.sun.ejb.containers.AbstractSingletonContainer$SingletonContextFactory.create(AbstractSingletonContainer.java:654)
      	at com.sun.ejb.containers.AbstractSingletonContainer.instantiateSingletonInstance(AbstractSingletonContainer.java:396)
      	- locked <0x000000075b3830d8> (a com.sun.ejb.containers.CMCSingletonContainer)
      	at org.glassfish.ejb.startup.SingletonLifeCycleManager.initializeSingleton(SingletonLifeCycleManager.java:219)
      	at org.glassfish.ejb.startup.SingletonLifeCycleManager.initializeSingleton(SingletonLifeCycleManager.java:180)
      	- locked <0x000000075b383358> (a org.glassfish.ejb.startup.SingletonLifeCycleManager)
      	at org.glassfish.ejb.startup.SingletonLifeCycleManager.doStartup(SingletonLifeCycleManager.java:158)
      	at org.glassfish.ejb.startup.EjbApplication.start(EjbApplication.java:166)
      	at org.glassfish.internal.data.EngineRef.start(EngineRef.java:122)
      	at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:291)
      	- locked <0x000000075b383478> (a org.glassfish.internal.data.ModuleInfo)
      	at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:352)
      	at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:497)
      	at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:407)
      	at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:243)
      	at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:281)
      	at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:328)
      	at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:448)
      	at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:163)
      	at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2204)
      	at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
      	- locked <0x000000075003f270> (a java.lang.Object)
      	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:673)
      	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:660)
      	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$UpOneJob.run(CurrentTaskFuture.java:490)
      	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$UpAllTheWay.go(CurrentTaskFuture.java:362)
      	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$UpAllTheWay.access$100(CurrentTaskFuture.java:279)
      	at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture.go(CurrentTaskFuture.java:113)
      	at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.proceedTo(AsyncRunLevelContext.java:296)
      	at org.glassfish.hk2.runlevel.internal.RunLevelControllerImpl.proceedTo(RunLevelControllerImpl.java:66)
      	at com.sun.enterprise.v3.server.AppServerStartup.proceedTo(AppServerStartup.java:532)
      	at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:329)
      	at com.sun.enterprise.v3.server.AppServerStartup.doStart(AppServerStartup.java:226)
      	at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:217)
      	- locked <0x000000075184dd50> (a com.sun.enterprise.v3.server.AppServerStartup)
      	at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79)
      	- locked <0x000000075184ddc0> (a com.sun.enterprise.glassfish.bootstrap.GlassFishImpl)
      	at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63)
      	at com.sun.enterprise.glassfish.bootstrap.osgi.EmbeddedOSGiGlassFishImpl.start(EmbeddedOSGiGlassFishImpl.java:75)
      	at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63)
      	at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishImpl.start(OSGiGlassFishImpl.java:71)
      	at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97)
      	at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:54)
      
      "VM Thread" prio=10 tid=0x00007f5d5c0eb800 nid=0xb0e runnable 
      
      "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f5d5c021800 nid=0xb07 runnable 
      
      "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f5d5c023800 nid=0xb08 runnable 
      
      "G1 Main Concurrent Mark GC Thread" prio=10 tid=0x00007f5d5c033800 nid=0xb0c runnable 
      
      "Gang worker#0 (G1 Parallel Marking Threads)" prio=10 tid=0x00007f5d5c03d800 nid=0xb0d runnable 
      
      "G1 Concurrent Refinement Thread#0" prio=10 tid=0x00007f5d5c029800 nid=0xb0b runnable 
      
      "G1 Concurrent Refinement Thread#1" prio=10 tid=0x00007f5d5c027800 nid=0xb0a runnable 
      
      "G1 Concurrent Refinement Thread#2" prio=10 tid=0x00007f5d5c026000 nid=0xb09 runnable 
      
      
      "VM Periodic Task Thread" prio=10 tid=0x00007f5d5c1a0000 nid=0xb16 waiting on condition 
      
      JNI global references: 427
      
      
      

      Workaround:

      There is a workaround. If the connection and send logic is run in a separate thread, the thread will lock until glassfish is started, then unlock and send the message

      @ApplicationScoped
      public class PerformanceLoggingProducer {
      
          private static final Logger logger = Logger.getLogger(PerformanceLoggingProducer.class.getName());
      
      //    @Resource(mappedName = "jms/ConnectionFactory")
          private ConnectionFactory connectionFactory;
      //    @Resource(name = "jms/PerformanceLoggingQueue")
          private Queue queue;
      
          public PerformanceLoggingProducer() {
              try {
                  connectionFactory = (ConnectionFactory) new InitialContext().lookup("jms/ConnectionFactory");
                  queue = (Queue) new InitialContext().lookup("jms/PerformanceLoggingQueue");
              } catch (NamingException ex) {
                  logger.log(Level.SEVERE, null, ex);
              }
          }
      
          public void log(final PerformanceLogTracker tracker) {
      
              if (tracker != null) {
            
                  Thread t = new Thread(new Runnable() {
      
                      @Override
                      public void run() {
                          try (Connection connection = connectionFactory.createConnection()) {
                              Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
                              MessageProducer producer = session.createProducer(queue);
                              ObjectMessage message = session.createObjectMessage(tracker);
                              producer.send(message);
                          } catch (Exception ex) {
                              logger.log(Level.SEVERE, "An error occurred when logging message: " + tracker, ex);
                          }
                      }
                  });
                  
                  t.start();
      
              }
          }
      
      }
      

        Activity

        Hide
        David Zhao added a comment - - edited

        Is it a timing issue? If sleeping a while (for example, 10 - 60 seconds) before creating connection in the @Singleton @Startup bean helps?

        Show
        David Zhao added a comment - - edited Is it a timing issue? If sleeping a while (for example, 10 - 60 seconds) before creating connection in the @Singleton @Startup bean helps?

          People

          • Assignee:
            David Zhao
            Reporter:
            electricsam
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated: