sailfin-cafe
  1. sailfin-cafe
  2. SAILFIN_CAFE-2

ConcurrentModificationError in nightly tests.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: current
    • Fix Version/s: milestone 1
    • Component/s: core
    • Labels:
      None
    • Environment:

      Operating System: All
      Platform: All

    • Issuezilla Id:
      2

      Description

      Nightly all of a sudden failed on test3. sipp started receiving 500 instead of 100 and the server.log
      contains the following.

      [#|2009-09-14T10:03:14.203-0700|INFO|sun-glassfish-comms-
      server2.0|cafe.test|_ThreadID=24;_ThreadName=SipContainer-serversWorkerThread-5060-
      5;|Conference Init ->conf1@example.com|#]

      [#|2009-09-14T10:03:14.297-0700|SEVERE|sun-glassfish-comms-
      server2.0|javax.enterprise.system.container.sip|_ThreadID=25;_ThreadName=SipContainer-
      serversWorkerThread-5060-8;_RequestID=a3f6ebcb-4c4f-42a1-a371-96b80877f13f;|Exception
      allocating servlet
      java.util.ConcurrentModificationException
      at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
      at java.util.HashMap$KeyIterator.next(HashMap.java:828)
      at org.glassfish.cafe.core.CommunicationImpl.getParticipant(CommunicationImpl.java:206)
      at org.glassfish.cafe.core.CafeBeanContainer.__runStateMachine(CafeBeanContainer.java:107)
      at org.glassfish.cafe.core.CafeBeanContainer.handleInvite(CafeBeanContainer.java:96)
      at org.glassfish.cafe.core.Servlet.doInvite(Servlet.java:36)
      at javax.servlet.sip.SipServlet.doRequest(SipServlet.java:57)
      at org.glassfish.cafe.core.Servlet.doRequest(Servlet.java:77)
      at javax.servlet.sip.SipServlet.service(SipServlet.java:46)
      at com.ericsson.ssa.container.sim.SipServletFacade.service(SipServletFacade.java:121)
      at com.ericsson.ssa.container.sim.SipServletWrapper.invokeServlet(SipServletWrapper.java:127)
      at com.ericsson.ssa.container.sim.ServletDispatcher.invoke(ServletDispatcher.java:291)
      at
      com.ericsson.ssa.container.sim.ApplicationDispatcher.invokeServletDispatchers(ApplicationDispatcher.j
      ava:365)
      at com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:174)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:53)
      at com.ericsson.ssa.container.sim.ApplicationDispatcher.next(ApplicationDispatcher.java:697)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.LocalRouteManager.next(LocalRouteManager.java:98)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.DialogManager.next(DialogManager.java:568)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.dns.ResolverManager.next(ResolverManager.java:169)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.persistence.ReplicationManager.next(ReplicationManager.java:102)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.sip.transaction.TransactionManager.invokeCreatedOrFetchedServerTransaction(Transa
      ctionManager.java:238)
      at com.ericsson.ssa.sip.transaction.TransactionManager.next(TransactionManager.java:291)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.container.OutboundFlowManager.processOutboundRequest(OutboundFlowManager.ja
      va:183)
      at com.ericsson.ssa.container.OutboundFlowManager.next(OutboundFlowManager.java:98)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.container.GrizzlyNetworkManager.next(GrizzlyNetworkManager.java:1331)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.container.MessageProcessorFilter.processMessage(MessageProcessorFilter.java:416)
      at com.ericsson.ssa.container.MessageProcessorFilter.execute(MessageProcessorFilter.java:303)
      at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:136)
      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:103)
      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:89)
      at
      com.ericsson.ssa.container.GrizzlyNetworkManager$SharedCallbackHandler.onRead(GrizzlyNetworkMan
      ager.java:3136)
      at com.sun.grizzly.CallbackHandlerContextTask.doCall(CallbackHandlerContextTask.java:76)
      at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:56)
      at com.sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:325)
      at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:184)

      #]

      [#|2009-09-14T10:03:14.304-0700|WARNING|sun-glassfish-comms-
      server2.0|javax.enterprise.system.container.sip|_ThreadID=25;_ThreadName=SipContainer-
      serversWorkerThread-5060-8;_RequestID=a3f6ebcb-4c4f-42a1-a371-96b80877f13f;|Error invoking
      service
      com.ericsson.ssa.container.sim.ServletInvocationException: java.util.ConcurrentModificationException
      at com.ericsson.ssa.container.sim.SipServletWrapper.invokeServlet(SipServletWrapper.java:134)
      at com.ericsson.ssa.container.sim.ServletDispatcher.invoke(ServletDispatcher.java:291)
      at
      com.ericsson.ssa.container.sim.ApplicationDispatcher.invokeServletDispatchers(ApplicationDispatcher.j
      ava:365)
      at com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:174)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:53)
      at com.ericsson.ssa.container.sim.ApplicationDispatcher.next(ApplicationDispatcher.java:697)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.LocalRouteManager.next(LocalRouteManager.java:98)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.DialogManager.next(DialogManager.java:568)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.dns.ResolverManager.next(ResolverManager.java:169)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.persistence.ReplicationManager.next(ReplicationManager.java:102)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.sip.transaction.TransactionManager.invokeCreatedOrFetchedServerTransaction(Transa
      ctionManager.java:238)
      at com.ericsson.ssa.sip.transaction.TransactionManager.next(TransactionManager.java:291)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.container.OutboundFlowManager.processOutboundRequest(OutboundFlowManager.ja
      va:183)
      at com.ericsson.ssa.container.OutboundFlowManager.next(OutboundFlowManager.java:98)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.container.GrizzlyNetworkManager.next(GrizzlyNetworkManager.java:1331)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.container.MessageProcessorFilter.processMessage(MessageProcessorFilter.java:416)
      at com.ericsson.ssa.container.MessageProcessorFilter.execute(MessageProcessorFilter.java:303)
      at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:136)
      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:103)
      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:89)
      at
      com.ericsson.ssa.container.GrizzlyNetworkManager$SharedCallbackHandler.onRead(GrizzlyNetworkMan
      ager.java:3136)
      at com.sun.grizzly.CallbackHandlerContextTask.doCall(CallbackHandlerContextTask.java:76)
      at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:56)
      at com.sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:325)
      at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:184)
      Caused by: java.util.ConcurrentModificationException
      at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
      at java.util.HashMap$KeyIterator.next(HashMap.java:828)
      at org.glassfish.cafe.core.CommunicationImpl.getParticipant(CommunicationImpl.java:206)
      at org.glassfish.cafe.core.CafeBeanContainer.__runStateMachine(CafeBeanContainer.java:107)
      at org.glassfish.cafe.core.CafeBeanContainer.handleInvite(CafeBeanContainer.java:96)
      at org.glassfish.cafe.core.Servlet.doInvite(Servlet.java:36)
      at javax.servlet.sip.SipServlet.doRequest(SipServlet.java:57)
      at org.glassfish.cafe.core.Servlet.doRequest(Servlet.java:77)
      at javax.servlet.sip.SipServlet.service(SipServlet.java:46)
      at com.ericsson.ssa.container.sim.SipServletFacade.service(SipServletFacade.java:121)
      at com.ericsson.ssa.container.sim.SipServletWrapper.invokeServlet(SipServletWrapper.java:127)
      ... 32 more

      #]

      [#|2009-09-14T10:08:14.294-0700|WARNING|sun-glassfish-comms-
      server2.0|javax.enterprise.resource.resourceadapter|_ThreadID=26;_ThreadName=Timer-
      205;_RequestID=b05de4a6-0597-40a0-9275-f3ab6c991f3a;|The log message is null.|#]

        Activity

        Hide
        binod added a comment -

        There are two issues here.

        1. The concurrent modification exception.

        That is a simple bug. Need to use java.util.concurrent for saving participants.

        2. Whether that is fixed or not, test3 some time fails. This happens in the following scenario. Since the
        conference is invoked from within the call's initialization event, some times (when all the parties reach
        the initialization event at the same time), the context/session thread local set on one thread for call
        event might be cleared by the conference. This creates unexpected behavior in the runtime.

        Fix is to introduce a nested thread local so that each nested event will use its own copy of these objects.

        Show
        binod added a comment - There are two issues here. 1. The concurrent modification exception. That is a simple bug. Need to use java.util.concurrent for saving participants. 2. Whether that is fixed or not, test3 some time fails. This happens in the following scenario. Since the conference is invoked from within the call's initialization event, some times (when all the parties reach the initialization event at the same time), the context/session thread local set on one thread for call event might be cleared by the conference. This creates unexpected behavior in the runtime. Fix is to introduce a nested thread local so that each nested event will use its own copy of these objects.
        Hide
        binod added a comment -

        Nested thread local has been introduced and is now fixed.

        Show
        binod added a comment - Nested thread local has been introduced and is now fixed.

          People

          • Assignee:
            binod
            Reporter:
            binod
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: