glassfish
  1. glassfish
  2. GLASSFISH-15288

Class not found exception from loading persistent state from a file from a previous run.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.1_b33
    • Fix Version/s: 3.1_b36
    • Component/s: deployment
    • Labels:
      None
    • Environment:

      solaris sparc

      Description

      Assigning to admin gui as this seems to be a gui issue that it is not prompting for a restart.

      build 33

      Deployed niles bookstore on a 3-node cluster.
      Using Oracle Load Testing Harness ran longevity test.

      bug: Class not found exception from loading persistent state from a file from a previous run.
      See EJBObjectInputStream.resolveClass() in stack trace below.

      #|2010-12-10T12:53:46.436-0800|SEVERE|oracle-glassfish3.1|org.apache.catalina.session.ManagerBase|_ThreadID=16;_ThreadName=Thread-1;|PWC2773: Exception loading sessions from persistent storage
      java.lang.ClassNotFoundException: NileBookStore.session.BigDummyObject

      This message occurs just BEFORE the deploying NileBookStore message.

      Full stack traces and deploy log message at end of this email at [1].

      This error would not happen if there had not been a start/stop/start cycle in testing.

      [1]/net/asqe-logs.us.oracle.com/export1/gms/gf31/gms/set_12_19_10_t_22_43_52/scenario_0001_Sun_Dec_19_22_44_16_PST_2010/n1c1m3/logs/server.log
      [#|2010-12-10T12:53:46.427-0800|SEVERE|oracle-glassfish3.1|org.apache.catalina.session.ManagerBase|_ThreadID=16;_ThreadName=Thread-1;|PWC2767: ClassNotFoundException while loading persisted sessions: java.lang.ClassNotFoundException: NileBookStore.session.BigDummyObject
      java.lang.ClassNotFoundException: NileBookStore.session.BigDummyObject
      at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
      at org.apache.felix.framework.ModuleImpl.doImplicitBootDelegation(ModuleImpl.java:1535)
      at org.apache.felix.framework.ModuleImpl.searchDynamicImports(ModuleImpl.java:1472)
      at org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:734)
      at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
      at org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:247)
      at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:603)
      at com.sun.ejb.base.io.EJBObjectInputStream.resolveClass(EJBObjectInputStream.java:163)
      at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1574)
      at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
      at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1623)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1322)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
      at org.apache.catalina.session.StandardSession.readRemainingObject(StandardSession.java:1951)
      at org.apache.catalina.session.StandardSession.readObject(StandardSession.java:1859)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
      at org.apache.catalina.session.StandardSession.deserialize(StandardSession.java:1144)
      at org.apache.catalina.session.StandardManager.readSessions(StandardManager.java:492)
      at org.apache.catalina.session.StandardManager.doLoadFromFile(StandardManager.java:422)
      at org.apache.catalina.session.StandardManager.load(StandardManager.java:393)
      at org.apache.catalina.session.StandardManager.start(StandardManager.java:834)
      at org.apache.catalina.core.StandardContext.start(StandardContext.java:5307)
      at com.sun.enterprise.web.WebModule.start(WebModule.java:497)
      at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:917)
      at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:901)
      at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:750)
      at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1979)
      at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1627)
      at com.sun.enterprise.web.WebApplication.start(WebApplication.java:100)
      at org.glassfish.internal.data.EngineRef.start(EngineRef.java:130)
      at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:269)
      at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:290)
      at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:461)
      at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:351)
      at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:202)
      at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:128)
      at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:88)
      at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:79)
      at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:64)
      at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:136)
      at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:73)
      at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:243)
      at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:135)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97)
      at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55)

      #]

      [#|2010-12-10T12:53:46.436-0800|SEVERE|oracle-glassfish3.1|org.apache.catalina.session.ManagerBase|_ThreadID=16;_ThreadName=Thread-1;|PWC2773: Exception loading sessions from persistent storage
      java.lang.ClassNotFoundException: NileBookStore.session.BigDummyObject
      at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
      at org.apache.felix.framework.ModuleImpl.doImplicitBootDelegation(ModuleImpl.java:1535)
      at org.apache.felix.framework.ModuleImpl.searchDynamicImports(ModuleImpl.java:1472)
      at org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:734)
      at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
      at org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:247)
      at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:603)
      at com.sun.ejb.base.io.EJBObjectInputStream.resolveClass(EJBObjectInputStream.java:163)
      at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1574)
      at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
      at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1623)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1322)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
      at org.apache.catalina.session.StandardSession.readRemainingObject(StandardSession.java:1951)
      at org.apache.catalina.session.StandardSession.readObject(StandardSession.java:1859)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
      at org.apache.catalina.session.StandardSession.deserialize(StandardSession.java:1144)
      at org.apache.catalina.session.StandardManager.readSessions(StandardManager.java:492)
      at org.apache.catalina.session.StandardManager.doLoadFromFile(StandardManager.java:422)
      at org.apache.catalina.session.StandardManager.load(StandardManager.java:393)
      at org.apache.catalina.session.StandardManager.start(StandardManager.java:834)
      at org.apache.catalina.core.StandardContext.start(StandardContext.java:5307)
      at com.sun.enterprise.web.WebModule.start(WebModule.java:497)
      at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:917)
      at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:901)
      at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:750)
      at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1979)
      at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1627)
      at com.sun.enterprise.web.WebApplication.start(WebApplication.java:100)
      at org.glassfish.internal.data.EngineRef.start(EngineRef.java:130)
      at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:269)
      at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:290)
      at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:461)
      at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:351)
      at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:202)
      at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:128)
      at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:88)
      at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:79)
      at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:64)
      at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:136)
      at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:73)
      at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:243)
      at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:135)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97)
      at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55)

      #]

      [#|2010-12-10T12:53:46.950-0800|INFO|oracle-glassfish3.1|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=1;_ThreadName=main;|WEB0671: Loading application [NileBookStore] at [/NileBookStore]|#]

      [#|2010-12-10T12:53:46.950-0800|INFO|oracle-glassfish3.1|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=16;_ThreadName=Thread-1;|WEB0671: Loading application [NileBookStore] at [/NileBookStore]|#]

      [#|2010-12-10T12:53:46.950-0800|INFO|oracle-glassfish3.1|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=1;_ThreadName=main;|WEB0671: Loading application [NileBookStore] at [/NileBookStore]|#]

      [2] /net/asqe-logs.us.oracle.com/export1/gms/gf31/gms/set_12_19_10_t_22_43_52/scenario_0001_Sun_Dec_19_22_44_16_PST_2010/n1c1m3/logs/server.log

      [#|2010-12-10T00:22:34.875-0800|WARNING|oracle-glassfish3.1|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=16;_ThreadName=Thread-1;|
      java.lang.IllegalStateException
      at com.sun.jts.jta.TransactionServiceProperties.initRecovery(TransactionServiceProperties.java:283)
      at com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate.initRecovery(JavaEETransactionManagerJTSDelegate.java:408)
      at com.sun.enterprise.transaction.JavaEETransactionManagerSimplified.initRecovery(JavaEETransactionManagerSimplified.java:287)
      at com.sun.enterprise.transaction.startup.TransactionRecoveryWrapper.onReady(TransactionRecoveryWrapper.java:100)
      at com.sun.enterprise.transaction.startup.TransactionRecoveryWrapper$1.event(TransactionRecoveryWrapper.java:86)
      at org.glassfish.kernel.event.EventsImpl.send(EventsImpl.java:128)
      at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:321)
      at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:135)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97)
      at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55)

      #]

      log:
      http://aras2.us.oracle.com:8080/logs/gf31/gms/set_12_19_10_t_22_43_52/scenario_0001_Sun_Dec_19_22_44_16_PST_2010/n1c1m3/logs/server.log

      all logs:
      http://aras2.us.oracle.com:8080/logs/gf31/gms/set_12_19_10_t_22_43_52/scenario_0001_Sun_Dec_19_22_44_16_PST_2010/

        Activity

        Hide
        Hong Zhang added a comment -

        Ok, I will work with the web team to see why the server restart status need to be set for setting the "availability-enabled" attribute.

        Zorro: can you please publish the complete steps/application for me to reproduce the problem?

        And a couple questions for the steps your described earlier:
        1) Via command line deployed application Nilebookstore.
        2) Via Admin GUI set the availabitiy to true.
        3) GUI DID NOT prompt for restarting the server and checking domain.xml showed that the availability is set to true.

        If in 2), you use asadmin CLI set command to set the availability attribute to true, does it make a difference?

        If you restart the cluster even if the GUI did not prompt for it, would things work as expected?

        I want to find out if there is a different between GUI and CLI on marking the server restart status, and also if restarting the cluster will indeed solve the problem. Thanks for your help.

        Show
        Hong Zhang added a comment - Ok, I will work with the web team to see why the server restart status need to be set for setting the "availability-enabled" attribute. Zorro: can you please publish the complete steps/application for me to reproduce the problem? And a couple questions for the steps your described earlier: 1) Via command line deployed application Nilebookstore. 2) Via Admin GUI set the availabitiy to true. 3) GUI DID NOT prompt for restarting the server and checking domain.xml showed that the availability is set to true. If in 2), you use asadmin CLI set command to set the availability attribute to true, does it make a difference? If you restart the cluster even if the GUI did not prompt for it, would things work as expected? I want to find out if there is a different between GUI and CLI on marking the server restart status, and also if restarting the cluster will indeed solve the problem. Thanks for your help.
        Hide
        Hong Zhang added a comment -

        After talking to Rajiv, the application/server restart is necessary to reload the application with HA enabled so the necessary HA set up could happen in the container code.
        There is currently an application config listener which listens to the application config related events and reload application as needed. I will make changes there to reload the application when the availability-enabled attribute is changed (this will work for both CLI and GUI paths).

        Show
        Hong Zhang added a comment - After talking to Rajiv, the application/server restart is necessary to reload the application with HA enabled so the necessary HA set up could happen in the container code. There is currently an application config listener which listens to the application config related events and reload application as needed. I will make changes there to reload the application when the availability-enabled attribute is changed (this will work for both CLI and GUI paths).
        Hide
        Hong Zhang added a comment -

        How bad is its impact? (Severity)
        Could be hard to debug as user will assume the HA is enabled for the application after setting the attribute.

        How often does it happen? Will many users see this problem? (Frequency)
        If user deploys the application without HA enabled initially, and later wants to take advantage of the HA by enabling the HA attribute (through CLI or GUI).

        How much effort is required to fix it? (Cost)
        The fix is available, waiting for QE to provide steps to verify.

        What is the risk of fixing it and how will the risk be mitigated? (Risk)
        Small. The fix is to register an additional config change with the application config change event listener to trigger the application reloading. Will run the necessary tests before checking in the change.

        Show
        Hong Zhang added a comment - How bad is its impact? (Severity) Could be hard to debug as user will assume the HA is enabled for the application after setting the attribute. How often does it happen? Will many users see this problem? (Frequency) If user deploys the application without HA enabled initially, and later wants to take advantage of the HA by enabling the HA attribute (through CLI or GUI). How much effort is required to fix it? (Cost) The fix is available, waiting for QE to provide steps to verify. What is the risk of fixing it and how will the risk be mitigated? (Risk) Small. The fix is to register an additional config change with the application config change event listener to trigger the application reloading. Will run the necessary tests before checking in the change.
        Hide
        Chris Kasso added a comment -

        Approved for 3.1

        Show
        Chris Kasso added a comment - Approved for 3.1
        Hide
        Hong Zhang added a comment -

        Talked to Shingwai about the set of the steps to reproduce and verify and he could not reproduce this in his environment. He thinks we could only reproduce this in the SQE environment. As the application gets reloaded now with my change after the attribute is set, I have gone ahead and checked in the changes and will let SQE verify in their environment.

        Zorro: could you please use tomorrow's nightly (12/30) or later builds to verify the fix. Thanks!

        Show
        Hong Zhang added a comment - Talked to Shingwai about the set of the steps to reproduce and verify and he could not reproduce this in his environment. He thinks we could only reproduce this in the SQE environment. As the application gets reloaded now with my change after the attribute is set, I have gone ahead and checked in the changes and will let SQE verify in their environment. Zorro: could you please use tomorrow's nightly (12/30) or later builds to verify the fix. Thanks!

          People

          • Assignee:
            Hong Zhang
            Reporter:
            zorro
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: