sailfin
  1. sailfin
  2. SAILFIN-1033

Stoping callflow monitoring causing few NPE / server freeze

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 1.0
    • Fix Version/s: milestone 1
    • Component/s: admin
    • Labels:
      None
    • Environment:

      Operating System: All
      Platform: All

      Description

      This is related to issue 1032. Stopping callflow monitoring has a few side effects.
      1. Few NPEs. Looks like this is causing some messages to be dropped.
      2. Also the CLI command takes a few minutes to complete.
      3. In some cases the server seems to completely freeze causing calls to backup
      in the sipp clients and eventually stop.

      [#|2008-07-11T21:22:07.017+0530|WARNING|sun-glassfish-comms-server0.5|javax.enterprise.system.tools.admin|_ThreadID=20;_ThreadName=SipContainer-serversWorkerThread-5060-5;_RequestID=e7ee4504-217b-4269-a6a2-2748646400c8;|ADM11020:
      Call flow web method end operation failed.
      java.lang.NullPointerException
      at
      com.sun.enterprise.admin.monitor.callflow.AgentImpl.storeMethodEndData(AgentImpl.java:1032)
      at
      com.sun.enterprise.admin.monitor.callflow.AgentImpl.webMethodEnd(AgentImpl.java:751)
      at
      org.jvnet.glassfish.comms.admin.callflow.reporter.CallflowReporter.reportPost(CallflowReporter.java:121)
      at
      com.ericsson.ssa.container.callflow.ComplexReporter.reportPost(ComplexReporter.java:93)
      at
      com.ericsson.ssa.container.sim.SipServletWrapper.invokeServlet(SipServletWrapper.java:153)
      at
      com.ericsson.ssa.container.sim.ServletDispatcher.invoke(ServletDispatcher.java:306)
      at
      com.ericsson.ssa.container.sim.ApplicationDispatcher.invokeServletDispatchers(ApplicationDispatcher.java:325)
      at
      com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:167)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:50)
      at
      com.ericsson.ssa.container.sim.ApplicationDispatcher.next(ApplicationDispatcher.java:527)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at com.ericsson.ssa.sip.LocalRouteManager.next(LocalRouteManager.java:104)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at com.ericsson.ssa.sip.DialogManager.next(DialogManager.java:487)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at com.ericsson.ssa.sip.dns.ResolverManager.next(ResolverManager.java:187)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at
      com.ericsson.ssa.sip.persistence.ReplicationManager.next(ReplicationManager.java:102)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at
      com.ericsson.ssa.sip.transaction.TransactionManager.invokeCreatedOrFetchedServerTransaction(TransactionManager.java:220)
      at
      com.ericsson.ssa.sip.transaction.TransactionManager.next(TransactionManager.java:271)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at
      com.ericsson.ssa.container.GrizzlyNetworkManager.next(GrizzlyNetworkManager.java:932)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at
      com.ericsson.ssa.container.MessageProcessorFilter.processMessage(MessageProcessorFilter.java:368)
      at
      com.ericsson.ssa.container.MessageProcessorFilter.access$000(MessageProcessorFilter.java:86)
      at
      com.ericsson.ssa.container.MessageProcessorFilter$1.call(MessageProcessorFilter.java:216)
      at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:154)

      #]

      [#|2008-07-11T21:22:07.016+0530|WARNING|sun-glassfish-comms-server0.5|javax.enterprise.system.tools.admin|_ThreadID=28;_ThreadName=SipContainer-serversWorkerThread-5060-1;_RequestID=518b76c1-9c50-498f-a07f-23934e0a4351;_ApplicationName=InviteServlet;_ModuleName=SIP
      Module;_ComponentName=InviteServlet;_ComponentType=SERVLET;_MethodName=service;_CallerId=anonymous;|ADM11016:
      Call flow end time operation failed.
      java.lang.NullPointerException
      at
      com.sun.enterprise.admin.monitor.callflow.AgentImpl.storeEndTimeData(AgentImpl.java:1061)
      at
      com.sun.enterprise.admin.monitor.callflow.AgentImpl.endTime(AgentImpl.java:530)
      at
      org.jvnet.glassfish.comms.admin.callflow.reporter.CallflowReporter.reportPost(CallflowReporter.java:78)
      at
      com.ericsson.ssa.container.callflow.ComplexReporter.reportPost(ComplexReporter.java:69)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.LocalRouteManager.next(LocalRouteManager.java:104)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at com.ericsson.ssa.sip.DialogManager.next(DialogManager.java:487)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at com.ericsson.ssa.sip.dns.ResolverManager.next(ResolverManager.java:187)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at
      com.ericsson.ssa.sip.persistence.ReplicationManager.next(ReplicationManager.java:102)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at
      com.ericsson.ssa.sip.transaction.TransactionManager.invokeCreatedOrFetchedServerTransaction(TransactionManager.java:220)
      at
      com.ericsson.ssa.sip.transaction.TransactionManager.next(TransactionManager.java:271)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at
      com.ericsson.ssa.container.GrizzlyNetworkManager.next(GrizzlyNetworkManager.java:932)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:57)
      at
      com.ericsson.ssa.container.MessageProcessorFilter.processMessage(MessageProcessorFilter.java:368)
      at
      com.ericsson.ssa.container.MessageProcessorFilter.access$000(MessageProcessorFilter.java:86)
      at
      com.ericsson.ssa.container.MessageProcessorFilter$1.call(MessageProcessorFilter.java:216)
      at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:154)

      #]

        Activity

        Hide
        sonymanuel added a comment -

        Created an attachment (id=550)
        server log

        Show
        sonymanuel added a comment - Created an attachment (id=550) server log
        Hide
        sanandal added a comment -

        Modifying keyword to shark-approved

        Show
        sanandal added a comment - Modifying keyword to shark-approved
        Hide
        sanandal added a comment -

        Modifying keyword to shark-approved

        Show
        sanandal added a comment - Modifying keyword to shark-approved
        Hide
        sanandal added a comment -

        update keyword

        Show
        sanandal added a comment - update keyword
        Hide
        Yamini K B added a comment -

        Unable to reproduce this problem at load of 50cps

        Show
        Yamini K B added a comment - Unable to reproduce this problem at load of 50cps
        Hide
        prasads added a comment -

        Marking as shark-approved

        Show
        prasads added a comment - Marking as shark-approved
        Hide
        Yamini K B added a comment -

        Have checked in the fix suggested by Harpreet to fix the freeze while disabling
        callflow.

        Have also raised a GlassFish issue 5357 for graceful handling of exceptions
        generated when disabling callflow under load.

        Show
        Yamini K B added a comment - Have checked in the fix suggested by Harpreet to fix the freeze while disabling callflow. Have also raised a GlassFish issue 5357 for graceful handling of exceptions generated when disabling callflow under load.

          People

          • Assignee:
            Yamini K B
            Reporter:
            sonymanuel
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: