sailfin
  1. sailfin
  2. SAILFIN-1992

Various ClassCastExceptions observed in the failed/restarted instance after restart in the conference scenario

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0
    • Fix Version/s: milestone 1
    • Component/s: session_replication
    • Labels:
      None
    • Environment:

      Operating System: All
      Platform: All

      Description

      Parent Issue: 1983
      Build: 31b 15th nightly.

      ClassCastException was observed for the following cases (Full Stack Traces are
      below the list):

      1. org.jvnet.glassfish.comms.replication.dialogmgmt.HADialogFragment cannot be
      cast to org.jvnet.glassfish.comms.replication.sessmgmt.HASipApplicationSession

      2. org.jvnet.glassfish.comms.replication.sessmgmt.FullHASipSession cannot be
      cast to org.jvnet.glassfish.comms.replication.sessmgmt.HASipApplicationSession

      3. org.jvnet.glassfish.comms.replication.sessmgmt.FullHASipSession cannot be
      cast to org.jvnet.glassfish.comms.replication.dialogmgmt.HADialogFragment

      4. org.jvnet.glassfish.comms.replication.sessmgmt.FullHASipApplicationSession
      cannot be cast to org.jvnet.glassfish.comms.replication.sessmgmt.HASipSession

      5. org.jvnet.glassfish.comms.replication.dialogmgmt.HADialogFragment cannot be
      cast to org.jvnet.glassfish.comms.replication.sessmgmt.HASipSession

      Complete Stack Traces:

      **********
      1.
      [#|2009-09-15T23:12:13.459+0530|SEVERE|sun-glassfish-comms-server2.0|org.apache.coyote.tomcat5.CoyoteAdapter|_ThreadID=34;_ThreadName=httpSSLWorkerThread-28080-1;_RequestID=25c1a3f0-f56a-459a-98a2-a1433fc9607a;|PWC3989:
      An exception or error occurred in the container during the request processing
      java.lang.ClassCastException:
      org.jvnet.glassfish.comms.replication.dialogmgmt.HADialogFragment cannot be cast
      to org.jvnet.glassfish.comms.replication.sessmgmt.HASipApplicationSession
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipApplicationSessionStoreImpl.getSipApplicationSession(SipApplicationSessionStoreImpl.java:443)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipApplicationSessionStoreImpl.getSipApplicationSession(SipApplicationSessionStoreImpl.java:377)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipApplicationSessionStoreImpl.loadFromRemoteActiveCache(SipApplicationSessionStoreImpl.java:185)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipApplicationSessionStoreImpl.load(SipApplicationSessionStoreImpl.java:153)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipApplicationSessionStoreImpl.load(SipApplicationSessionStoreImpl.java:120)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.swapInSipApplicationSession(SipTransactionPersistentManager.java:4976)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.swapInSipApplicationSession(SipTransactionPersistentManager.java:4930)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.swapInSipApplicationSession(SipTransactionPersistentManager.java:4905)
      at
      com.ericsson.ssa.sip.PersistentSipSessionManagerBase.findSipApplicationSession(PersistentSipSessionManagerBase.java:253)
      at
      com.ericsson.ssa.sip.PersistentSipSessionManagerBase.findSipApplicationSession(PersistentSipSessionManagerBase.java:210)
      at
      com.ericsson.ssa.sip.ConvergedHttpSessionFacade.getApplicationSession(ConvergedHttpSessionFacade.java:170)
      at
      com.ericsson.ssa.config.ConvergedContextImpl.beginPipelineInvoke(ConvergedContextImpl.java:606)
      at
      com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:96)
      at
      com.sun.enterprise.ee.web.sessmgmt.SessionLockingStandardPipeline.invoke(SessionLockingStandardPipeline.java:120)
      at
      org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:222)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
      at
      org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:587)
      at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1093)
      at
      org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:166)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
      at
      org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:587)
      at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1093)
      at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:291)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.invokeAdapter(DefaultProcessorTask.java:666)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.doProcess(DefaultProcessorTask.java:597)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.process(DefaultProcessorTask.java:872)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultReadTask.executeProcessorTask(DefaultReadTask.java:341)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:263)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:214)
      at
      org.jvnet.glassfish.comms.clb.proxy.portunif.ClbProxyPipeline$PUTask.doTask(ClbProxyPipeline.java:532)
      at com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:264)
      at
      com.sun.enterprise.web.connector.grizzly.ssl.SSLWorkerThread.run(SSLWorkerThread.java:106)

      #]
      *************

      2.

      [#|2009-09-15T23:12:13.563+0530|SEVERE|sun-glassfish-comms-server2.0|javax.enterprise.system.container.sip|_ThreadID=36;_ThreadName=SipContainer-serversWorkerThread-5060-0;_RequestID=d3edd186-dd88-4dc6-a0ee-c92b33a2c501;|An
      exception occurred during request processing.
      java.lang.ClassCastException:
      org.jvnet.glassfish.comms.replication.sessmgmt.FullHASipSession cannot be cast
      to org.jvnet.glassfish.comms.replication.sessmgmt.HASipApplicationSession
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipApplicationSessionStoreImpl.getSipApplicationSession(SipApplicationSessionStoreImpl.java:443)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipApplicationSessionStoreImpl.getSipApplicationSession(SipApplicationSessionStoreImpl.java:377)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipApplicationSessionStoreImpl.loadFromRemoteActiveCache(SipApplicationSessionStoreImpl.java:185)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipApplicationSessionStoreImpl.load(SipApplicationSessionStoreImpl.java:153)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipApplicationSessionStoreImpl.load(SipApplicationSessionStoreImpl.java:120)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.swapInSipApplicationSession(SipTransactionPersistentManager.java:4976)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.swapInSipApplicationSession(SipTransactionPersistentManager.java:4918)
      at
      com.ericsson.ssa.sip.PersistentSipSessionManagerBase.findSipApplicationSession(PersistentSipSessionManagerBase.java:243)
      at
      com.ericsson.ssa.sip.PersistentSipSessionManagerBase.findSipApplicationSession(PersistentSipSessionManagerBase.java:210)
      at
      com.ericsson.ssa.sip.SipSessionDialogImpl.getPFieldSipApplicationSession(SipSessionDialogImpl.java:565)
      at
      com.ericsson.ssa.sip.SipSessionImplBase.getApplicationSession(SipSessionImplBase.java:473)
      at
      com.ericsson.ssa.sip.persistence.ReplicationUnitOfWork.lockDialog(ReplicationUnitOfWork.java:185)
      at
      com.ericsson.ssa.sip.dialog.DialogLifeCycle.initUnitOfWork(DialogLifeCycle.java:265)
      at
      com.ericsson.ssa.sip.dialog.DialogLifeCycle.initUnitOfWork(DialogLifeCycle.java:247)
      at com.ericsson.ssa.sip.DialogManager.getSession(DialogManager.java:1121)
      at
      com.ericsson.ssa.sip.DialogManager.setDialogContext(DialogManager.java:412)
      at com.ericsson.ssa.sip.DialogManager.next(DialogManager.java:485)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.dns.ResolverManager.next(ResolverManager.java:169)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.sip.persistence.ReplicationManager.next(ReplicationManager.java:102)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.sip.transaction.TransactionManager.invokeCreatedOrFetchedServerTransaction(TransactionManager.java:238)
      at
      com.ericsson.ssa.sip.transaction.TransactionManager.next(TransactionManager.java:291)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      org.jvnet.glassfish.comms.clb.core.sip.SipLoadBalancerManagerBackEnd.next(SipLoadBalancerManagerBackEnd.java:193)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      org.jvnet.glassfish.comms.clb.core.sip.SipLoadBalancerManager.next(SipLoadBalancerManager.java:157)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.container.OutboundFlowManager.processOutboundRequest(OutboundFlowManager.java:183)
      at
      com.ericsson.ssa.container.OutboundFlowManager.next(OutboundFlowManager.java:98)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.container.GrizzlyNetworkManager.next(GrizzlyNetworkManager.java:1331)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.container.MessageProcessorFilter.processMessage(MessageProcessorFilter.java:416)
      at
      com.ericsson.ssa.container.MessageProcessorFilter.execute(MessageProcessorFilter.java:315)
      at
      com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:136)
      at
      com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:103)
      at
      com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:89)
      at
      com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:67)
      at
      com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:56)
      at
      com.sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:325)
      at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:184)

      #]

      *****************

      3.

      [#|2009-09-15T23:12:13.825+0530|SEVERE|sun-glassfish-comms-server2.0|javax.enterprise.system.container.sip|_ThreadID=38;_ThreadName=SipContainer-serversWorkerThread-5060-3;_RequestID=4c9cf56f-b117-49ce-a1d6-9b452f970cc4;|An
      exception occurred during request processing.
      java.lang.ClassCastException:
      org.jvnet.glassfish.comms.replication.sessmgmt.FullHASipSession cannot be cast
      to org.jvnet.glassfish.comms.replication.dialogmgmt.HADialogFragment
      at
      org.jvnet.glassfish.comms.replication.dialogmgmt.DialogFragmentStoreImpl.getDialogFragment(DialogFragmentStoreImpl.java:454)
      at
      org.jvnet.glassfish.comms.replication.dialogmgmt.DialogFragmentStoreImpl.getDialogFragment(DialogFragmentStoreImpl.java:406)
      at
      org.jvnet.glassfish.comms.replication.dialogmgmt.DialogFragmentStoreImpl.loadFromRemoteActiveCache(DialogFragmentStoreImpl.java:171)
      at
      org.jvnet.glassfish.comms.replication.dialogmgmt.DialogFragmentStoreImpl.load(DialogFragmentStoreImpl.java:138)
      at
      org.jvnet.glassfish.comms.replication.dialogmgmt.DialogFragmentStoreImpl.load(DialogFragmentStoreImpl.java:113)
      at
      org.jvnet.glassfish.comms.replication.dialogmgmt.ReplicationDialogFragmentManager.swapInDialogFragment(ReplicationDialogFragmentManager.java:982)
      at
      org.jvnet.glassfish.comms.replication.dialogmgmt.ReplicationDialogFragmentManager.swapInDialogFragment(ReplicationDialogFragmentManager.java:943)
      at
      com.ericsson.ssa.sip.PersistentDialogFragmentManager.findDialogFragment(PersistentDialogFragmentManager.java:93)
      at com.ericsson.ssa.sip.DialogManager.getSession(DialogManager.java:1079)
      at
      com.ericsson.ssa.sip.DialogManager.setDialogContext(DialogManager.java:412)
      at com.ericsson.ssa.sip.DialogManager.next(DialogManager.java:485)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at com.ericsson.ssa.sip.dns.ResolverManager.next(ResolverManager.java:169)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.sip.persistence.ReplicationManager.next(ReplicationManager.java:102)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.sip.transaction.TransactionManager.invokeCreatedOrFetchedServerTransaction(TransactionManager.java:238)
      at
      com.ericsson.ssa.sip.transaction.TransactionManager.next(TransactionManager.java:291)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      org.jvnet.glassfish.comms.clb.core.sip.SipLoadBalancerManagerBackEnd.next(SipLoadBalancerManagerBackEnd.java:193)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      org.jvnet.glassfish.comms.clb.core.sip.SipLoadBalancerManager.next(SipLoadBalancerManager.java:157)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.container.OutboundFlowManager.processOutboundRequest(OutboundFlowManager.java:183)
      at
      com.ericsson.ssa.container.OutboundFlowManager.next(OutboundFlowManager.java:98)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.container.GrizzlyNetworkManager.next(GrizzlyNetworkManager.java:1331)
      at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:60)
      at
      com.ericsson.ssa.container.MessageProcessorFilter.processMessage(MessageProcessorFilter.java:416)
      at
      com.ericsson.ssa.container.MessageProcessorFilter.execute(MessageProcessorFilter.java:315)
      at
      com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:136)
      at
      com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:103)
      at
      com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:89)
      at
      com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:67)
      at
      com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:56)
      at
      com.sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:325)
      at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:184)

      #]

      ***************

      4.

      [#|2009-09-15T23:12:14.402+0530|WARNING|sun-glassfish-comms-server2.0|javax.enterprise.system.container.sip|_ThreadID=34;_ThreadName=httpSSLWorkerThrea
      d-28080-1;_RequestID=25c1a3f0-f56a-459a-98a2-a1433fc9607a;|ss id =
      15,15,conf5061_168572conference
      java.lang.RuntimeException: ss id = 15,15,conf5061_168572conference
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.getSipSession(SipSessionStoreImpl.java:355)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.loadFromRemoteActiveCache(SipSessionStoreImpl.java:164)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.load(SipSessionStoreImpl.java:131)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.load(SipSessionStoreImpl.java:118)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.swapInSipSession(SipTransactionPersistentManager.java:5341)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.swapInSipSession(SipTransactionPersistentManager.java:5296)
      at
      com.ericsson.ssa.sip.PersistentSipSessionManagerBase.findSipSession(PersistentSipSessionManagerBase.java:113)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.findSipSession(SipTransactionPersistentManager.java:2960)
      at
      com.ericsson.ssa.sip.SipApplicationSessionImpl$1$1.hasNext(SipApplicationSessionImpl.java:387)
      at
      com.ericsson.ssa.sip.SipApplicationSessionImpl.hasValidSipSession(SipApplicationSessionImpl.java:556)
      at
      com.ericsson.ssa.sip.SipApplicationSessionImpl.isEmpty(SipApplicationSessionImpl.java:590)
      at
      com.ericsson.ssa.sip.SipApplicationSessionBase.isReadyToInvalidate(SipApplicationSessionBase.java:954)
      at
      com.ericsson.ssa.sip.SipApplicationSessionBase.checkSessionReadyToInvalidate(SipApplicationSessionBase.java:968)
      at
      com.ericsson.ssa.sip.SipApplicationSessionImpl.removePFieldHttpProtocolSession(SipApplicationSessionImpl.java:755)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.HASipApplicationSession.removePFieldHttpProtocolSession(HASipApplicationSession.java:652)
      at
      com.ericsson.ssa.sip.SipApplicationSessionBase.removeSession(SipApplicationSessionBase.java:729)
      at
      com.ericsson.ssa.sip.ConvergedHttpSessionFacade.removeFromSAS(ConvergedHttpSessionFacade.java:110)
      at
      com.ericsson.ssa.sip.ConvergedHttpSessionFacade.invalidate(ConvergedHttpSessionFacade.java:99)
      at
      com.sun.asqe.systemtest.conference.http.ConferenceHTTPServlet.processFinalPage(ConferenceHTTPServlet.java:238)
      at
      com.sun.asqe.systemtest.conference.http.ConferenceHTTPServlet.processRequest(ConferenceHTTPServlet.java:44)
      at
      com.sun.asqe.systemtest.conference.http.ConferenceHTTPServlet.doPost(ConferenceHTTPServlet.java:262)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
      at
      org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:427)
      at
      org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:315)
      at
      org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:287)
      at
      org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:218)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
      at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:94)
      at
      com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:98)
      at
      com.sun.enterprise.ee.web.sessmgmt.SessionLockingStandardPipeline.invoke(SessionLockingStandardPipeline.java:120)
      at
      org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:222)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
      at
      org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:587)
      at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1093)
      at
      org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:166)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
      at
      org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:587)
      at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1093)
      at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:291)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.invokeAdapter(DefaultProcessorTask.java:666)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.doProcess(DefaultProcessorTask.java:597)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.process(DefaultProcessorTask.java:872)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultReadTask.executeProcessorTask(DefaultReadTask.java:341)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:263)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:214)
      at
      org.jvnet.glassfish.comms.clb.proxy.portunif.ClbProxyPipeline$PUTask.doTask(ClbProxyPipeline.java:532)
      at com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:264)
      at
      com.sun.enterprise.web.connector.grizzly.ssl.SSLWorkerThread.run(SSLWorkerThread.java:106)Caused
      by: java.lang.ClassCastException:
      org.jvnet.glassfish.comms.replication.sessmgmt.FullHASipApplicationSession
      cannot be cast to org.jvnet.glassfish.comms.replication.sessmgmt.HASipSession
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.getSipSession(SipSessionStoreImpl.java:418)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.getSipSession(SipSessionStoreImpl.java:350)
      ... 51 more

      #]

      ******************

      5.

      [#|2009-09-15T23:12:16.707+0530|WARNING|sun-glassfish-comms-server2.0|javax.enterprise.system.container.sip|_ThreadID=34;_ThreadName=httpSSLWorkerThrea
      d-28080-1;_RequestID=25c1a3f0-f56a-459a-98a2-a1433fc9607a;|ss id =
      164793-18922@10.12.156.128&164793&fzmxhnh8-3dp2:instance109_1
      java.lang.RuntimeException: ss id =
      164793-18922@10.12.156.128&164793&fzmxhnh8-3dp2:instance109_1
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.getSipSession(SipSessionStoreImpl.java:355)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.loadFromRemoteActiveCache(SipSessionStoreImpl.java:164)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.load(SipSessionStoreImpl.java:131)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.load(SipSessionStoreImpl.java:118)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.swapInSipSession(SipTransactionPersistentManager.java:5341)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.swapInSipSession(SipTransactionPersistentManager.java:5296)
      at
      com.ericsson.ssa.sip.PersistentSipSessionManagerBase.findSipSession(PersistentSipSessionManagerBase.java:113)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipTransactionPersistentManager.findSipSession(SipTransactionPersistentManager.java:2960)
      at
      com.ericsson.ssa.sip.SipApplicationSessionImpl$1$1.hasNext(SipApplicationSessionImpl.java:387)
      at
      com.ericsson.ssa.sip.SipApplicationSessionImpl.hasValidSipSession(SipApplicationSessionImpl.java:556)
      at
      com.ericsson.ssa.sip.SipApplicationSessionImpl.isEmpty(SipApplicationSessionImpl.java:590)
      at
      com.ericsson.ssa.sip.SipApplicationSessionBase.isReadyToInvalidate(SipApplicationSessionBase.java:954)
      at
      com.ericsson.ssa.sip.SipApplicationSessionBase.checkSessionReadyToInvalidate(SipApplicationSessionBase.java:968)
      at
      com.ericsson.ssa.sip.SipApplicationSessionImpl.removePFieldHttpProtocolSession(SipApplicationSessionImpl.java:755)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.HASipApplicationSession.removePFieldHttpProtocolSession(HASipApplicationSession.java:652)
      at
      com.ericsson.ssa.sip.SipApplicationSessionBase.removeSession(SipApplicationSessionBase.java:729)
      at
      com.ericsson.ssa.sip.ConvergedHttpSessionFacade.removeFromSAS(ConvergedHttpSessionFacade.java:110)
      at
      com.ericsson.ssa.sip.ConvergedHttpSessionFacade.invalidate(ConvergedHttpSessionFacade.java:99)
      at
      com.sun.asqe.systemtest.conference.http.ConferenceHTTPServlet.processFinalPage(ConferenceHTTPServlet.java:238)
      at
      com.sun.asqe.systemtest.conference.http.ConferenceHTTPServlet.processRequest(ConferenceHTTPServlet.java:44)
      at
      com.sun.asqe.systemtest.conference.http.ConferenceHTTPServlet.doPost(ConferenceHTTPServlet.java:262)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
      at
      org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:427)
      at
      org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:315)
      at
      org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:287)
      at
      org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:218)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
      at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:94)
      at
      com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:98)
      at
      com.sun.enterprise.ee.web.sessmgmt.SessionLockingStandardPipeline.invoke(SessionLockingStandardPipeline.java:120)
      at
      org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:222)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
      at
      org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:587)
      at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1093)
      at
      org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:166)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
      at
      org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
      at
      org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:587)
      at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1093)
      at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:291)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.invokeAdapter(DefaultProcessorTask.java:666)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.doProcess(DefaultProcessorTask.java:597)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.process(DefaultProcessorTask.java:872)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultReadTask.executeProcessorTask(DefaultReadTask.java:341)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:263)
      at
      com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:214)
      at
      org.jvnet.glassfish.comms.clb.proxy.portunif.ClbProxyPipeline$PUTask.doTask(ClbProxyPipeline.java:532)
      at com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:264)
      at
      com.sun.enterprise.web.connector.grizzly.ssl.SSLWorkerThread.run(SSLWorkerThread.java:106)
      Caused by: java.lang.ClassCastException:
      org.jvnet.glassfish.comms.replication.dialogmgmt.HADialogFragment cannot be cast
      to org.jvnet.glassfish.comms.replication.sessmgmt.HASipSession
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.getSipSession(SipSessionStoreImpl.java:418)
      at
      org.jvnet.glassfish.comms.replication.sessmgmt.SipSessionStoreImpl.getSipSession(SipSessionStoreImpl.java:350)
      ... 51 more

      #]

      *****************

        Issue Links

          Activity

          Hide
          varunrupela added a comment -

          updated keyword

          Show
          varunrupela added a comment - updated keyword
          Hide
          Bhavanishankar added a comment -

          Hi Varun,

          We need the following informations to debug this issue:

          (a) Are you seeing this only after restarting the failed instance? If so, then
          is this seen only on the restarted instance?

          (b) Can you attach all the server.log* & domain.xml from all instances?

          Show
          Bhavanishankar added a comment - Hi Varun, We need the following informations to debug this issue: (a) Are you seeing this only after restarting the failed instance? If so, then is this seen only on the restarted instance? (b) Can you attach all the server.log* & domain.xml from all instances?
          Hide
          varunrupela added a comment -
          • Yes, the ClassCastExceptions are only on the restarted instance and only after
            restart.
          • Please check the logs at:
            sf-x2200-11.india.sun.com:/space/sony/logs/2.0/b31b-15thnightly/conference-ssr/IEC1-SuSE
          Show
          varunrupela added a comment - Yes, the ClassCastExceptions are only on the restarted instance and only after restart. Please check the logs at: sf-x2200-11.india.sun.com:/space/sony/logs/2.0/b31b-15thnightly/conference-ssr/IEC1-SuSE
          Hide
          varunrupela added a comment -

          Created an attachment (id=1120)
          attached required logs

          Show
          varunrupela added a comment - Created an attachment (id=1120) attached required logs
          Hide
          Mahesh Kannan added a comment -

          Scott did the analysis and here is comment:
          <scott>
          The ReplicationResponseRepository has a map of queues. When you have a request,
          the process is this:

          ReplicationResponseRepository.putEmptyQueueEntry(state);
          // That assigns a queue based on state.id()
          send state
          ReplicationResponseRepositry.getEntry()
          // That gets the queue
          // Polls the queue waiting for a response for the state.id
          // Then removes the queue from the map and recycles the queue

          Now consider what happens if two threads at the same time (e.g. the HTTP and SIP
          threads in the converged app) request the same id at the same time. They both
          call putEmptyQueueEntry(state) which assigns a queue to the id. Now the first
          thread to get a response will remove the queue from the map.

          Two things happen now:

          1) The second thread that had made the response will get back null when it looks
          up the queue inside getEntry(). So it will think that the object isn't found. I
          believe that is what causes the error where the SAS can't be found after failure
          in the Converged app.

          2) The queue is recycled, so another lookup will use the queue. Perhaps the
          previous lookup was for a SAS and the new lookup is for an SS. Unfortunately,
          when the 2nd SAS response comes in, it is put onto the queue where someone is
          now waiting for an SS. That causes the classcastexception.

          </scott>

          We have a patch (including the deadlock patch) that fixes this issue. I am
          currently running this patch on a 10 instance cluster and even with a higher cps
          (75) haven't see the ClassCast so far.

          Show
          Mahesh Kannan added a comment - Scott did the analysis and here is comment: <scott> The ReplicationResponseRepository has a map of queues. When you have a request, the process is this: ReplicationResponseRepository.putEmptyQueueEntry(state); // That assigns a queue based on state.id() send state ReplicationResponseRepositry.getEntry() // That gets the queue // Polls the queue waiting for a response for the state.id // Then removes the queue from the map and recycles the queue Now consider what happens if two threads at the same time (e.g. the HTTP and SIP threads in the converged app) request the same id at the same time. They both call putEmptyQueueEntry(state) which assigns a queue to the id. Now the first thread to get a response will remove the queue from the map. Two things happen now: 1) The second thread that had made the response will get back null when it looks up the queue inside getEntry(). So it will think that the object isn't found. I believe that is what causes the error where the SAS can't be found after failure in the Converged app. 2) The queue is recycled, so another lookup will use the queue. Perhaps the previous lookup was for a SAS and the new lookup is for an SS. Unfortunately, when the 2nd SAS response comes in, it is put onto the queue where someone is now waiting for an SS. That causes the classcastexception. </scott> We have a patch (including the deadlock patch) that fixes this issue. I am currently running this patch on a 10 instance cluster and even with a higher cps (75) haven't see the ClassCast so far.
          Hide
          Mahesh Kannan added a comment -

          Fixed.

          proton-2% cvs commit -m "Fix for 1992. Sailfin and replication passed"
          ReplicationResponseRepository.java ReplicationResponseRepositoryEntry.java
          Checking in ReplicationResponseRepository.java;
          /cvs/glassfish/appserv-core-ee/http-session-persistence/src/java/com/sun/enterprise/ee/web/sessmgmt/ReplicationResponseRepository.java,v
          <-- ReplicationResponseRepository.java
          new revision: 1.7.2.7.6.1; previous revision: 1.7.2.7
          done
          Checking in ReplicationResponseRepositoryEntry.java;
          /cvs/glassfish/appserv-core-ee/http-session-persistence/src/java/com/sun/enterprise/ee/web/sessmgmt/ReplicationResponseRepositoryEntry.java,v
          <-- ReplicationResponseRepositoryEntry.java
          new revision: 1.3.22.1; previous revision: 1.3
          done

          Show
          Mahesh Kannan added a comment - Fixed. proton-2% cvs commit -m "Fix for 1992. Sailfin and replication passed" ReplicationResponseRepository.java ReplicationResponseRepositoryEntry.java Checking in ReplicationResponseRepository.java; /cvs/glassfish/appserv-core-ee/http-session-persistence/src/java/com/sun/enterprise/ee/web/sessmgmt/ReplicationResponseRepository.java,v <-- ReplicationResponseRepository.java new revision: 1.7.2.7.6.1; previous revision: 1.7.2.7 done Checking in ReplicationResponseRepositoryEntry.java; /cvs/glassfish/appserv-core-ee/http-session-persistence/src/java/com/sun/enterprise/ee/web/sessmgmt/ReplicationResponseRepositoryEntry.java,v <-- ReplicationResponseRepositoryEntry.java new revision: 1.3.22.1; previous revision: 1.3 done
          Hide
          varunrupela added a comment -

          Verified with Build 31c

          Show
          varunrupela added a comment - Verified with Build 31c

            People

            • Assignee:
              Mahesh Kannan
              Reporter:
              varunrupela
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: