grizzly
  1. grizzly
  2. GRIZZLY-1173

sync thread pool is getting overflowed

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.7
    • Fix Version/s: 2.1.9, 2.2.1
    • Component/s: framework
    • Labels:
      None
    • Environment:

      Linux 2.4.21-37.ELsmp #1 SMP Wed Sep 7 13:24:00 EDT 2005 i686 athlon i386 GNU/Linux ; mvn3 - jdk6

      Description

      Logging to the "admin console" of GlassFish (from a Quicklook test) sometime fails with the following stack trace:

          java.net.SocketException: Connection reset
          	at java.net.SocketInputStream.read(SocketInputStream.java:168)
          	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
          	at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
          	at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78)
          	at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106)
          	at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
          	at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973)
          	at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
          	at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
          	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
          	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
          	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
          	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
          	at test.admin.BaseAdminConsoleTest.getUrl(BaseAdminConsoleTest.java:209)
          	at test.admin.BaseAdminConsoleTest.getUrlAndTestForStrings(BaseAdminConsoleTest.java:187)
          	at test.admin.BaseAdminConsoleTest.loginBeforeTest(BaseAdminConsoleTest.java:130)
      

      Here is a snippet of the server.log:

          [#|2012-01-09T09:14:28.618-0800|SEVERE|44.0|org.glassfish.grizzly.nio.SelectorRunner|_ThreadID=29;_ThreadName=Thread-2;|doSelect exception
          java.util.concurrent.RejectedExecutionException: The thread pool's task queue is full, limit: 256
          	at org.glassfish.grizzly.threadpool.AbstractThreadPool.onTaskQueueOverflow(AbstractThreadPool.java:425)
          	at org.glassfish.grizzly.threadpool.SyncThreadPool.execute(SyncThreadPool.java:125)
          	at org.glassfish.grizzly.threadpool.GrizzlyExecutorService.execute(GrizzlyExecutorService.java:161)
          	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.executeIoEvent(WorkerThreadIOStrategy.java:94)
          	at org.glassfish.grizzly.nio.SelectorRunner.iterateKeyEvents(SelectorRunner.java:390)
          	at org.glassfish.grizzly.nio.SelectorRunner.iterateKeys(SelectorRunner.java:360)
          	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:326)
          	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:262)
          	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:508)
          	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:488)
          	at java.lang.Thread.run(Thread.java:662)
          |#]
      

      Investigated with Oleksei. Some more glassfish thread dump is available in the attached file ("serverlog-pool-overflow.txt").

        Activity

        Hide
        oleksiys added a comment -

        fixed.

        thanks.

        Project: grizzly
        Repository: git
        Revision: 9c31ca7b85eda82aaffee3ffc7f75f755144ccfa
        Author: oleksiys
        Date: 2012-01-10 16:05:02 UTC
        Link:

        Log Message:
        ------------
        + fix issue #1173
        http://java.net/jira/browse/GRIZZLY-1173
        "sync thread pool is getting overflowed"

        Show
        oleksiys added a comment - fixed. thanks. Project: grizzly Repository: git Revision: 9c31ca7b85eda82aaffee3ffc7f75f755144ccfa Author: oleksiys Date: 2012-01-10 16:05:02 UTC Link: Log Message: ------------ + fix issue #1173 http://java.net/jira/browse/GRIZZLY-1173 "sync thread pool is getting overflowed"
        Hide
        oleksiys added a comment -

        update the fix versions

        Show
        oleksiys added a comment - update the fix versions
        Hide
        Romain Grécourt added a comment -

        I've reproduced the bug with the patched artifact you gave me. After 12 trials, the 13th failed with the same error, but in a different test: still part of the admingui tests but not as part of the initialization... The failing tests is "testCommonTasks"...

        Alex, can you re-open the issue, I don't have the rights to do so. Thanks.

        Show
        Romain Grécourt added a comment - I've reproduced the bug with the patched artifact you gave me. After 12 trials, the 13th failed with the same error, but in a different test: still part of the admingui tests but not as part of the initialization... The failing tests is "testCommonTasks"... Alex, can you re-open the issue, I don't have the rights to do so. Thanks.
        Hide
        oleksiys added a comment -

        Hi Romain,

        the Grizzly part of issue is fixed, but looks like admin console initialization just requires larger thread-pool queue, pls. increase it. Currently the default max queue size is 256 (see GF's domain.xml)
        <thread-pool name="admin-thread-pool" max-thread-pool-size="50" max-queue-size="256"></thread-pool>

        (for some reason it's set explicitly for admin-thread-pool), if you remove the max-queue-size attribute like:
        <thread-pool name="admin-thread-pool" max-thread-pool-size="50"></thread-pool>

        the default max-queue-size will be 4096, which probably will be enough for admin console.

        WBR.

        Show
        oleksiys added a comment - Hi Romain, the Grizzly part of issue is fixed, but looks like admin console initialization just requires larger thread-pool queue, pls. increase it. Currently the default max queue size is 256 (see GF's domain.xml) <thread-pool name="admin-thread-pool" max-thread-pool-size="50" max-queue-size="256" ></thread-pool> (for some reason it's set explicitly for admin-thread-pool), if you remove the max-queue-size attribute like: <thread-pool name="admin-thread-pool" max-thread-pool-size="50"></thread-pool> the default max-queue-size will be 4096, which probably will be enough for admin console. WBR.

          People

          • Assignee:
            Unassigned
            Reporter:
            Romain Grécourt
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: