glassfish
  1. glassfish
  2. GLASSFISH-17385

asadmin stop-domain does not end cleanly for MDB's

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.1_dev
    • Fix Version/s: 4.1.1
    • Component/s: ejb_container
    • Labels:
      None

      Description

      When trying to shut down glassfish using the command:

      asadmin stop-domain --force=false

      Looking at our logs, our Message Driven Beans will throw the following error when trying to finish processing by retrieving the messages that have been placed on the queue:

      onMessage(): Failed to process JMS message.
      javax.ejb.EJBException: Attempt to invoke when container is in STOPPED
      at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1995) [ejb-container.jar:3.1.1]
      at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1990) [ejb-container.jar:3.1.1]

      When stopping our domain, we are unable to finish processing gracefully if this exception occurs.

      Please advise.

      Thanks!

        Activity

        Hide
        tchng added a comment -

        The feature we are looking for is:

        "shutdown when work completes"

        Show
        tchng added a comment - The feature we are looking for is: "shutdown when work completes"
        Hide
        tchng added a comment - - edited

        This exception behavior still happens in Glassfish version 3.2-b06
        15:20:17.360 [30b96561-d0d5-4cd8-a36d-dd8a5a610afb] [p: thread-pool-1; w: 17] ERROR c.b.z.s.w.SyncWorkflowMessageListener - onMessage(): Failed to process JMS message.
        javax.ejb.EJBException: Attempt to invoke when container is in STOPPED
        at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1996) [ejb-container.jar:3.2-b06]
        at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1991) [ejb-container.jar:3.2-b06]
        at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:222) ~[ejb-container.jar:3.2-b06]
        at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88) ~[ejb-container.jar:3.2-b06]
        il.reflection.SecureReflectionAccess.run(SecureReflectionAccess.java:54) ~[na:na]
        at org.jboss.weld.util.reflection.SecureReflectionAccess.runAsInvocation(SecureReflectionAccess.java:163) ~[na:na]
        at org.jboss.weld.util.reflection.SecureReflections.invoke(SecureReflections.java:299) ~[na:na]

        Show
        tchng added a comment - - edited This exception behavior still happens in Glassfish version 3.2-b06 15:20:17.360 [30b96561-d0d5-4cd8-a36d-dd8a5a610afb] [p: thread-pool-1; w: 17] ERROR c.b.z.s.w.SyncWorkflowMessageListener - onMessage(): Failed to process JMS message. javax.ejb.EJBException: Attempt to invoke when container is in STOPPED at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1996) [ejb-container.jar:3.2-b06] at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1991) [ejb-container.jar:3.2-b06] at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:222) ~ [ejb-container.jar:3.2-b06] at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88) ~ [ejb-container.jar:3.2-b06] il.reflection.SecureReflectionAccess.run(SecureReflectionAccess.java:54) ~ [na:na] at org.jboss.weld.util.reflection.SecureReflectionAccess.runAsInvocation(SecureReflectionAccess.java:163) ~ [na:na] at org.jboss.weld.util.reflection.SecureReflections.invoke(SecureReflections.java:299) ~ [na:na]
        Hide
        tchng added a comment -

        We are able to simulate this problem beyond mdb's

        Using a facelet page, that hits a JSF Managed bean which then iterates 40 times with a 1 second sleep hitting an EJB session bean, we then invoke:

        asadmin stop-domain --force=false

        The result of this creates the same exception:

        [#|2011-10-12T14:58:09.808+0000|SEVERE|glassfish3.2|javax.enterprise.resource.webcontainer.jsf.application|_ThreadID=41;_ThreadName=Thread-2;|Error Rendering View[/index.xhtml]
        javax.el.ELException: /index.xhtml: javax.ejb.EJBException: Attempt to invoke when container is in STOPPED
        at com.sun.faces.facelets.compiler.TextInstruction.write(TextInstruction.java:88)
        at com.sun.faces.facelets.compiler.UIInstructions.encodeBegin(UIInstructions.java:82)
        at com.sun.faces.facelets.compiler.UILeaf.encodeAll(UILeaf.java:183)
        at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1759)
        at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1759)
        at com.sun.faces.application.view.FaceletViewHandlingStrategy.renderView(FaceletViewHandlingStrategy.java:401)
        at com.sun.faces.application.view.MultiViewHandler.renderView(MultiViewHandler.java:131)
        at javax.faces.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:288)
        at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:121)
        at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
        at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:410)
        at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1562)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:286)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:173)
        at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)
        at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:98)
        at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:91)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:162)
        at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:345)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:242)
        at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:172)
        at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:162)
        at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:160)
        at org.glassfish.grizzly.filterchain.ExecutorResolver$3.execute(ExecutorResolver.java:95)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:444)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:364)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:290)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:133)
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:76)
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:63)
        at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:823)
        at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:116)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$000(WorkerThreadIOStrategy.java:55)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$1.run(WorkerThreadIOStrategy.java:98)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:508)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:488)
        at java.lang.Thread.run(Thread.java:680)
        Caused by: javax.ejb.EJBException: Attempt to invoke when container is in STOPPED
        at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1996)
        at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1991)
        at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:222)
        at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
        at $Proxy313.getCounterValue(Unknown Source)
        at com.test.bean._EJB31_GeneratedCounterSessionBeanIntf__Bean_.getCounterValue(Unknown Source)
        at com.test.MyJSFManagedBean.getCount(MyJSFManagedBean.java:32)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at javax.el.BeanELResolver.getValue(BeanELResolver.java:302)
        at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:176)
        at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:203)
        at com.sun.el.parser.AstValue.getValue(AstValue.java:116)
        at com.sun.el.parser.AstValue.getValue(AstValue.java:163)
        at com.sun.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:219)
        at org.jboss.weld.el.WeldValueExpression.getValue(WeldValueExpression.java:55)
        at com.sun.faces.facelets.el.ELText$ELTextVariable.writeText(ELText.java:224)
        at com.sun.faces.facelets.el.ELText$ELTextComposite.writeText(ELText.java:148)
        at com.sun.faces.facelets.compiler.TextInstruction.write(TextInstruction.java:85)

        It seems that the challenge is that when the web container invokes upon the ejb container while we are seeking to stop glassfish gracefully, this exception occurs which is not a graceful shut down.

        The facelet skeleton project is an attachment.

        Show
        tchng added a comment - We are able to simulate this problem beyond mdb's Using a facelet page, that hits a JSF Managed bean which then iterates 40 times with a 1 second sleep hitting an EJB session bean, we then invoke: asadmin stop-domain --force=false The result of this creates the same exception: [#|2011-10-12T14:58:09.808+0000|SEVERE|glassfish3.2|javax.enterprise.resource.webcontainer.jsf.application|_ThreadID=41;_ThreadName=Thread-2;|Error Rendering View [/index.xhtml] javax.el.ELException: /index.xhtml: javax.ejb.EJBException: Attempt to invoke when container is in STOPPED at com.sun.faces.facelets.compiler.TextInstruction.write(TextInstruction.java:88) at com.sun.faces.facelets.compiler.UIInstructions.encodeBegin(UIInstructions.java:82) at com.sun.faces.facelets.compiler.UILeaf.encodeAll(UILeaf.java:183) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1759) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1759) at com.sun.faces.application.view.FaceletViewHandlingStrategy.renderView(FaceletViewHandlingStrategy.java:401) at com.sun.faces.application.view.MultiViewHandler.renderView(MultiViewHandler.java:131) at javax.faces.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:288) at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:121) at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101) at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:410) at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1562) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:286) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:173) at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595) at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:98) at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:91) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:162) at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:345) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:242) at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:172) at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:162) at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:160) at org.glassfish.grizzly.filterchain.ExecutorResolver$3.execute(ExecutorResolver.java:95) at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:444) at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:364) at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:290) at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:133) at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:76) at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:63) at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:823) at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113) at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:116) at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$000(WorkerThreadIOStrategy.java:55) at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$1.run(WorkerThreadIOStrategy.java:98) at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:508) at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:488) at java.lang.Thread.run(Thread.java:680) Caused by: javax.ejb.EJBException: Attempt to invoke when container is in STOPPED at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1996) at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:1991) at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:222) at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88) at $Proxy313.getCounterValue(Unknown Source) at com.test.bean._ EJB31_Generated CounterSessionBean Intf __ Bean _.getCounterValue(Unknown Source) at com.test.MyJSFManagedBean.getCount(MyJSFManagedBean.java:32) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at javax.el.BeanELResolver.getValue(BeanELResolver.java:302) at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:176) at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:203) at com.sun.el.parser.AstValue.getValue(AstValue.java:116) at com.sun.el.parser.AstValue.getValue(AstValue.java:163) at com.sun.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:219) at org.jboss.weld.el.WeldValueExpression.getValue(WeldValueExpression.java:55) at com.sun.faces.facelets.el.ELText$ELTextVariable.writeText(ELText.java:224) at com.sun.faces.facelets.el.ELText$ELTextComposite.writeText(ELText.java:148) at com.sun.faces.facelets.compiler.TextInstruction.write(TextInstruction.java:85) It seems that the challenge is that when the web container invokes upon the ejb container while we are seeking to stop glassfish gracefully, this exception occurs which is not a graceful shut down. The facelet skeleton project is an attachment.
        Hide
        tchng added a comment - - edited

        The facelet skeleton project that simulates this problem which occurs during the exection of:

        asadmin stop-domain --force=false

        That if a thread in the web-container is invoking something in the ejb-container in glassfish, we get the exception: javax.ejb.EJBException: Attempt to invoke when container is in STOPPED

        Which when executing an mdb, the thread is expected to finish cleanly.

        Show
        tchng added a comment - - edited The facelet skeleton project that simulates this problem which occurs during the exection of: asadmin stop-domain --force=false That if a thread in the web-container is invoking something in the ejb-container in glassfish, we get the exception: javax.ejb.EJBException: Attempt to invoke when container is in STOPPED Which when executing an mdb, the thread is expected to finish cleanly.
        Hide
        Satish Kumar added a comment -

        Reassigning to Marina since this issue is related to the MDB container...

        Show
        Satish Kumar added a comment - Reassigning to Marina since this issue is related to the MDB container...
        Hide
        Cheng Fang added a comment -

        A somewhat related issue is 17315 (cannot call EJB method from method sessionDestroyed (session listener) when web container wants undeploy enterprise application.), which has to do with the stop ordering between ejb container and web container.

        Graceful shutdown of ejb container involves coordiation with other modules (JMS, JCA, etc). Also need to understand the level of gracefulness, e.g., do we still accept new method invocation belonging to the same txn, the same session, and shutdown timeout. Exclude it from 3.1.2 release.

        Show
        Cheng Fang added a comment - A somewhat related issue is 17315 (cannot call EJB method from method sessionDestroyed (session listener) when web container wants undeploy enterprise application.), which has to do with the stop ordering between ejb container and web container. Graceful shutdown of ejb container involves coordiation with other modules (JMS, JCA, etc). Also need to understand the level of gracefulness, e.g., do we still accept new method invocation belonging to the same txn, the same session, and shutdown timeout. Exclude it from 3.1.2 release.

          People

          • Assignee:
            marina vatkina
            Reporter:
            tchng
          • Votes:
            5 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated: