glassfish
  1. glassfish
  2. GLASSFISH-17499

Uncaught NullPointerException during http.ProcessorTask.doProcess can break postProcess buffer cleanup

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.1.1_b12
    • Fix Version/s: 3.1.2_b08, 4.0
    • Component/s: web_container
    • Labels:
      None

      Description

      We are running the GlassFish Server version 3.1.1 build 12 plus patches loaded from the private Oracle repository on 8/16/2011.
      During testing we noted that the following pair of errors occurred following a very large (over 8192 byte) URI sent to the server. Thereafter, each time that the thread was reused, it generated the same pair of errors, most likely because the second error was not caught and caused the thread to be released without the postProcess buffer cleanup 'recycle()'. With just a few retries of the long URI, we managed to disable all of the domain's request processing threads.

      The error was triggered by the buffer overflow and then the unhandled null pointer error in the ErrorHandler caused the rest - the stack traces are shown below.

      The actual error appears to be in the ErrorHandler property of the associated thread; however the grizzly http.ProcessorTask.doProcess probably should protect itself.

      [#|2011-10-06T17:13:28.188-0400|SEVERE|oracle-glassfish3.1.1|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=28;_ThreadName=Thread-3;|GRIZZLY0039: Request URI is too large.
      java.nio.BufferOverflowException
      at com.sun.grizzly.tcp.http11.InternalInputBuffer.fill(InternalInputBuffer.java:765)
      at com.sun.grizzly.tcp.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:402)
      at com.sun.grizzly.http.ProcessorTask.parseRequest(ProcessorTask.java:861)
      at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:692)
      at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019)
      at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
      at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
      at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
      at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
      at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
      at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
      at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
      at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
      at java.lang.Thread.run(Thread.java:662)

      #]

      [#|2011-10-06T17:13:28.188-0400|SEVERE|oracle-glassfish3.1.1|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=28;_ThreadName=Thread-3;|GRIZZLY0051: ProcessorTask exception.
      java.lang.NullPointerException
      at java.nio.CharBuffer.put(CharBuffer.java:896)
      at com.sun.enterprise.web.accesslog.DefaultAccessLogFormatterImpl.appendRequestInfo(DefaultAccessLogFormatterImpl.java:494)
      at com.sun.enterprise.web.accesslog.DefaultAccessLogFormatterImpl.appendLogEntry(DefaultAccessLogFormatterImpl.java:264)
      at com.sun.enterprise.web.PEAccessLogValve.postInvoke(PEAccessLogValve.java:592)
      at com.sun.enterprise.web.VirtualServer$2.onParsingError(VirtualServer.java:1698)
      at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:709)
      at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019)
      at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
      at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
      at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
      at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
      at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
      at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
      at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
      at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
      at java.lang.Thread.run(Thread.java:662)

      #]

        Activity

        Hide
        gpambrozio added a comment -

        Is there a workaround for this issue? Maybe adding a filter to every request? I started seeing this error today on my servers and I'm worried it might be some kind of DOS or that it might become one...

        Show
        gpambrozio added a comment - Is there a workaround for this issue? Maybe adding a filter to every request? I started seeing this error today on my servers and I'm worried it might be some kind of DOS or that it might become one...
        Hide
        Amy Roh added a comment -

        Fixed in svn 50549 & 50551.

        Show
        Amy Roh added a comment - Fixed in svn 50549 & 50551.
        Hide
        Shing Wai Chan added a comment -

        The NPE is due to the fact that hreq.getRequestURI() is null.
        The URI is null because of the buffer overflow.

        Assign to Amy to investigate what should be done in this case.
        Note that one need to be able to identify this from the log later.

        Show
        Shing Wai Chan added a comment - The NPE is due to the fact that hreq.getRequestURI() is null. The URI is null because of the buffer overflow. Assign to Amy to investigate what should be done in this case. Note that one need to be able to identify this from the log later.
        Hide
        oleksiys added a comment -

        reassign to webcontainer to fix NPE.

        Show
        oleksiys added a comment - reassign to webcontainer to fix NPE.

          People

          • Assignee:
            Amy Roh
            Reporter:
            steve_cochran
          • Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: