glassfish
  1. glassfish
  2. GLASSFISH-18636

Unable to run Trade2 benchmark due to orb errors

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.0_b31
    • Fix Version/s: 4.0_b35
    • Component/s: ejb_container
    • Labels:
      None

      Description

      From b31 onwards we have noticed following exceptions in server.log file while running Trade2 performance benchmark. Due to this error Trade2 benchmark does not proceed as expected,

      [#|2012-04-09T00:36:09.229-0700|WARNING|44.0|javax.enterprise.system.container.ejb.com.sun.ejb.containers|_ThreadID=48;_ThreadName=Thread-2;|javax.ejb.TransactionRolledbackLocalException: Client's
      transaction aborted
      at com.sun.ejb.containers.BaseContainer.useClientTx(BaseContainer.java:4656)
      at com.sun.ejb.containers.BaseContainer.preInvokeTx(BaseContainer.java:4535)
      at com.sun.ejb.containers.BaseContainer.preInvoke(BaseContainer.java:1907)
      at com.sun.ejb.containers.EJBObjectInvocationHandler.invoke(EJBObjectInvocationHandler.java:205)
      at com.sun.ejb.containers.EJBObjectInvocationHandler.invoke(EJBObjectInvocationHandler.java:123)
      at $Proxy197.login(Unknown Source)
      at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:601)
      at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.privateInvoke(StubInvocationHandlerImpl.java:239)
      at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.invoke(StubInvocationHandlerImpl.java:150)
      at com.sun.corba.ee.impl.presentation.rmi.codegen.CodegenStubBase.invoke(CodegenStubBase.java:227)
      at trade._Trade_DynamicStub.login(trade/_Trade_DynamicStub.java)
      at trade.TradeAccessBean.login(TradeAccessBean.java:182)
      at trade_client.TradeAction.doLogin(TradeAction.java:311)
      at trade_client.TradeServletAction.doLogin(TradeServletAction.java:595)
      at trade_client.TradeScenarioServlet.performTask(TradeScenarioServlet.java:224)
      at trade_client.TradeScenarioServlet.doGet(TradeScenarioServlet.java:72)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
      at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1589)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:286)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:173)
      at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)
      at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)
      at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:337)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:240)
      at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:172)
      at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:163)
      at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:164)
      at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:265)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:200)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:134)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
      at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:78)
      at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:816)
      at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
      at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115)
      at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55)
      at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135)
      at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:567)
      at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:547)
      at java.lang.Thread.run(Thread.java:722)

      #]

      [#|2012-04-09T00:36:09.230-0700|INFO|44.0|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=41;_ThreadName=Thread-2;|javax.transaction.TransactionRolledbackException: CORBA TR
      ANSACTION_ROLLEDBACK 9998 Maybe; nested exception is:
      org.omg.CORBA.TRANSACTION_ROLLEDBACK: vmcid: 0x2000 minor code: 1806 completed: Maybe
      at com.sun.corba.ee.impl.javax.rmi.CORBA.Util.mapSystemException(Util.java:276)
      at com.sun.corba.ee.impl.javax.rmi.CORBA.Util.wrapException(Util.java:695)
      at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.privateInvoke(StubInvocationHandlerImpl.java:249)
      at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.invoke(StubInvocationHandlerImpl.java:150)
      at com.sun.corba.ee.impl.presentation.rmi.codegen.CodegenStubBase.invoke(CodegenStubBase.java:227)
      at trade._Trade_DynamicStub.login(trade/_Trade_DynamicStub.java)
      at trade.TradeAccessBean.login(TradeAccessBean.java:182)
      at trade_client.TradeAction.doLogin(TradeAction.java:311)
      at trade_client.TradeServletAction.doLogin(TradeServletAction.java:595)
      at trade_client.TradeScenarioServlet.performTask(TradeScenarioServlet.java:224)
      at trade_client.TradeScenarioServlet.doGet(TradeScenarioServlet.java:72)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
      at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1589)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:286)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:173)
      at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)
      at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)
      at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:337)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:240)
      at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:172)
      at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:163)
      at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:164)
      at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:265)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:200)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:134)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
      at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:78)
      at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:816)
      at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
      at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115)
      at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55)
      at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135)
      at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:567)
      at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:547)
      at java.lang.Thread.run(Thread.java:722)
      Caused by: org.omg.CORBA.TRANSACTION_ROLLEDBACK: vmcid: 0x2000 minor code: 1806 completed: Maybe
      Caused by: javax.ejb.TransactionRolledbackLocalException: Client's transaction aborted
      at com.sun.ejb.containers.BaseContainer.useClientTx(BaseContainer.java:4656)
      at com.sun.ejb.containers.BaseContainer.preInvokeTx(BaseContainer.java:4535)
      at com.sun.ejb.containers.BaseContainer.preInvoke(BaseContainer.java:1907)
      at com.sun.ejb.containers.EJBObjectInvocationHandler.invoke(EJBObjectInvocationHandler.java:205)
      at com.sun.ejb.containers.EJBObjectInvocationHandler.invoke(EJBObjectInvocationHandler.java:123)
      at $Proxy197.login(Unknown Source)
      at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:601)
      at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.privateInvoke(StubInvocationHandlerImpl.java:239)
      at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.invoke(StubInvocationHandlerImpl.java:150)
      at com.sun.corba.ee.impl.presentation.rmi.codegen.CodegenStubBase.invoke(CodegenStubBase.java:227)
      at trade._Trade_DynamicStub.login(trade/_Trade_DynamicStub.java)
      at trade.TradeAccessBean.login(TradeAccessBean.java:182)
      at trade_client.TradeAction.doLogin(TradeAction.java:311)
      at trade_client.TradeServletAction.doLogin(TradeServletAction.java:595)
      at trade_client.TradeScenarioServlet.performTask(TradeScenarioServlet.java:224)
      at trade_client.TradeScenarioServlet.doGet(TradeScenarioServlet.java:72)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
      at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1589)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:286)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:173)
      at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)
      at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)
      at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:337)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:240)
      at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:172)
      at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:163)
      at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:164)
      at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:265)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:200)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:134)
      at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
      at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:78)
      at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:816)
      at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
      at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115)
      at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55)
      at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135)
      at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:567)
      at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:547)
      at java.lang.Thread.run(Thread.java:722)

      #]

        Activity

        Hide
        marina vatkina added a comment -

        Can you try the latest nightly? Are there any other exceptions in the server.log before the TransactionRolledbackLocalException?

        Show
        marina vatkina added a comment - Can you try the latest nightly? Are there any other exceptions in the server.log before the TransactionRolledbackLocalException?
        Hide
        amitagarwal added a comment -

        We see this issue with latest build b32 as well, so I feel this will be with latest nightly too.
        I have attached a copy of server.log for your reference.

        Show
        amitagarwal added a comment - We see this issue with latest build b32 as well, so I feel this will be with latest nightly too. I have attached a copy of server.log for your reference.
        Hide
        amitagarwal added a comment -

        Forgot to mentioned that build b30 was fine, but build b31 onwards we started seeing this issue.

        Show
        amitagarwal added a comment - Forgot to mentioned that build b30 was fine, but build b31 onwards we started seeing this issue.
        Hide
        marina vatkina added a comment -

        Can you attach the rotated logs as well, so that I can possibly see the starting point of these exceptions?

        Can you attach the TradeBean.java?

        Show
        marina vatkina added a comment - Can you attach the rotated logs as well, so that I can possibly see the starting point of these exceptions? Can you attach the TradeBean.java?
        Hide
        amitagarwal added a comment -

        There are lots of rotated log files. You can browse them through this link,
        http://alacrity.us.oracle.com/net/alacrity.us.oracle.com/export/multicomplex/logs/p/4.0se.b31_Ois64/trade2_2se.20120408_231320/results.trade2_2se/results_4/output/1646/logs/perf-x4150-2/

        We don't have source code of Trade2 benchmark. And TradeBean.java is part of it.

        Show
        amitagarwal added a comment - There are lots of rotated log files. You can browse them through this link, http://alacrity.us.oracle.com/net/alacrity.us.oracle.com/export/multicomplex/logs/p/4.0se.b31_Ois64/trade2_2se.20120408_231320/results.trade2_2se/results_4/output/1646/logs/perf-x4150-2/ We don't have source code of Trade2 benchmark. And TradeBean.java is part of it.
        Hide
        marina vatkina added a comment -

        Is there a ejb-jar.xml?

        Show
        marina vatkina added a comment - Is there a ejb-jar.xml?
        Hide
        marina vatkina added a comment -

        The recent changes in the EJB Container should not affect SLSBs (which TradeBean is according to google searches (see. e.g. http://svn.apache.org/repos/asf/geronimo/daytrader/branches/m2standard/daytrader-ejb/src/main/resources/META-INF/ejb-jar.xml).
        Requesting web container team to check if any transactional support had changed from b31 onward.

        Show
        marina vatkina added a comment - The recent changes in the EJB Container should not affect SLSBs (which TradeBean is according to google searches (see. e.g. http://svn.apache.org/repos/asf/geronimo/daytrader/branches/m2standard/daytrader-ejb/src/main/resources/META-INF/ejb-jar.xml ). Requesting web container team to check if any transactional support had changed from b31 onward.
        Hide
        Shing Wai Chan added a comment -

        b30 is on Mar 28. There is no traditional changes in web container.
        From stack trace, it may be related to ejb container or transactions.

        Show
        Shing Wai Chan added a comment - b30 is on Mar 28. There is no traditional changes in web container. From stack trace, it may be related to ejb container or transactions.
        Hide
        amitagarwal added a comment -
        Show
        amitagarwal added a comment - You can access the .ear file here to check xmls and jars. http://alacrity.us.oracle.com/net/alacrity/export/multicomplex/workload/workload/benchmarks/trade2/jars/v3/Trade2.ear
        Hide
        marina vatkina added a comment -

        Can you run this ear without multi-threaded access? Will you see the same error? Or is there a readme how to run it locally?

        Show
        marina vatkina added a comment - Can you run this ear without multi-threaded access? Will you see the same error? Or is there a readme how to run it locally?
        Hide
        amitagarwal added a comment -

        I tried this with one user and get same exceptions, but some user requests get served too.
        Configuring this locally requires lots of steps and db installation too. Is it ok if we share our setup with you for few hours for debugging? Please let us know? You can coordinate with Scott too in US timezone.

        Show
        amitagarwal added a comment - I tried this with one user and get same exceptions, but some user requests get served too. Configuring this locally requires lots of steps and db installation too. Is it ok if we share our setup with you for few hours for debugging? Please let us know? You can coordinate with Scott too in US timezone.
        Hide
        Scott Oaks added a comment -

        Fixed in b35.

        Show
        Scott Oaks added a comment - Fixed in b35.

          People

          • Assignee:
            marina vatkina
            Reporter:
            amitagarwal
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: