glassfish
  1. glassfish
  2. GLASSFISH-18924

IOP01210054 durign (re)deployment of OSGi bundle during cluster instance (re)start

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.2
    • Fix Version/s: None
    • Component/s: OSGi
    • Labels:
      None

      Description

      I know that GLASSFISH-17971 should've resolved the problem but I see that it does not
      (and simply cannot) solve the problem in this case.

      The exception I get is this:

      IOP01210054: Error while attempting to load class com.sun.corba.ee.impl.oa.rfm.ReferenceManagerConfigurator
      org.omg.CORBA.BAD_OPERATION: WARNING: IOP01210054: Error while attempting to load class com.sun.corba.ee.impl.oa.rfm.ReferenceManagerConfigurator  vmcid: OMG  minor code: 54  completed: No
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
              at com.sun.corba.ee.spi.orbutil.logex.corba.CorbaExtension.makeException(CorbaExtension.java:248)
              at com.sun.corba.ee.spi.orbutil.logex.corba.CorbaExtension.makeException(CorbaExtension.java:95)
              at com.sun.corba.ee.spi.orbutil.logex.WrapperGenerator.handleFullLogging(WrapperGenerator.java:387)
              at com.sun.corba.ee.spi.orbutil.logex.WrapperGenerator.access$400(WrapperGenerator.java:107)
              at com.sun.corba.ee.spi.orbutil.logex.WrapperGenerator$2.invoke(WrapperGenerator.java:511)
              at com.sun.corba.ee.spi.orbutil.proxy.CompositeInvocationHandlerImpl.invoke(CompositeInvocationHandlerImpl.java:99)
              at $Proxy166.classActionException(Unknown Source)
              at com.sun.corba.ee.spi.orb.OperationFactory$ClassAction.operate(OperationFactory.java:297)
              at com.sun.corba.ee.spi.orb.OperationFactory$ComposeAction.operate(OperationFactory.java:520)
              at com.sun.corba.ee.impl.orb.PrefixParserAction.apply(PrefixParserAction.java:96)
              at com.sun.corba.ee.spi.orb.PropertyParser.parse(PropertyParser.java:84)
              at com.sun.corba.ee.spi.orb.ParserImplBase.init(ParserImplBase.java:77)
              at com.sun.corba.ee.impl.orb.ORBConfiguratorImpl.runUserConfigurators(ORBConfiguratorImpl.java:179)
              at com.sun.corba.ee.impl.orb.ORBConfiguratorImpl.configure(ORBConfiguratorImpl.java:170)
              at com.sun.corba.ee.impl.orb.ORBImpl.postInit(ORBImpl.java:625)
              at com.sun.corba.ee.impl.orb.ORBImpl.set_parameters(ORBImpl.java:704)
              at com.sun.corba.ee.impl.orb.ORBImpl.setParameters(ORBImpl.java:691)
              at com.sun.corba.ee.spi.osgi.ORBFactory.initialize(ORBFactory.java:107)
              at org.glassfish.enterprise.iiop.impl.GlassFishORBManager.initORB(GlassFishORBManager.java:581)
              at org.glassfish.enterprise.iiop.impl.GlassFishORBManager.getORB(GlassFishORBManager.java:263)
              at org.glassfish.enterprise.iiop.impl.GlassFishORBFactoryImpl.createORB(GlassFishORBFactoryImpl.java:93)
              at org.glassfish.enterprise.iiop.api.GlassFishORBHelper.getORB(GlassFishORBHelper.java:152)
              at org.glassfish.enterprise.iiop.api.GlassFishORBHelper.getProtocolManager(GlassFishORBHelper.java:219)
              at com.sun.ejb.containers.BaseContainer.initializeProtocolManager(BaseContainer.java:824)
              at com.sun.ejb.containers.BaseContainer.<init>(BaseContainer.java:508)
              at com.sun.ejb.containers.MessageBeanContainer.<init>(MessageBeanContainer.java:142)
              at com.sun.ejb.containers.ContainerFactoryImpl.createContainer(ContainerFactoryImpl.java:121)
              at org.glassfish.ejb.startup.EjbApplication.loadContainers(EjbApplication.java:230)
              at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:299)
              at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:105)
              at org.glassfish.internal.data.ModuleInfo.load(ModuleInfo.java:186)
              at org.glassfish.internal.data.ApplicationInfo.load(ApplicationInfo.java:264)
              at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:460)
              at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:240)
              at org.glassfish.osgijavaeebase.OSGiDeploymentRequest.deploy(OSGiDeploymentRequest.java:183)
              at org.glassfish.osgijavaeebase.OSGiDeploymentRequest.execute(OSGiDeploymentRequest.java:118)
              at org.glassfish.osgijavaeebase.AbstractOSGiDeployer.deploy(AbstractOSGiDeployer.java:121)
              at org.glassfish.osgijavaeebase.OSGiContainer.deploy(OSGiContainer.java:154)
              at org.glassfish.osgijavaeebase.OSGiContainer.redeploy(OSGiContainer.java:113)
              at org.glassfish.osgijavaeebase.OSGiContainer.access$500(OSGiContainer.java:61)
              at org.glassfish.osgijavaeebase.OSGiContainer$DeployerAddedThread.run(OSGiContainer.java:305)
      Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: com.sun.corba.ee.impl.oa.rfm.ReferenceManagerConfigurator
              at com.sun.corba.ee.spi.orb.ORB$2.evaluate(ORB.java:882)
              at com.sun.corba.ee.spi.orb.ORB$2.evaluate(ORB.java:877)
              at com.sun.corba.ee.spi.orb.ORB$3.evaluate(ORB.java:904)
              at com.sun.corba.ee.spi.orb.ORB$3.evaluate(ORB.java:900)
              at com.sun.corba.ee.spi.orb.OperationFactory$ClassAction.operate(OperationFactory.java:294)
              ... 33 more
      Caused by: java.lang.ClassNotFoundException: com.sun.corba.ee.impl.oa.rfm.ReferenceManagerConfigurator
              at java.lang.ClassLoader.findClass(ClassLoader.java:373)
              at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
              at com.sun.enterprise.v3.server.APIClassLoaderServiceImpl$APIClassLoader.loadClass(APIClassLoaderServiceImpl.java:206)
              at com.sun.enterprise.v3.server.APIClassLoaderServiceImpl$APIClassLoader.loadClass(APIClassLoaderServiceImpl.java:182)
              at org.glassfish.osgiejb.OSGiEJBDeploymentContext$DelegatingInstrumentableClassLoader.loadClass(OSGiEJBDeploymentContext.java:101)
              at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
              at com.sun.corba.ee.spi.orb.ORB$2.evaluate(ORB.java:880)
              ... 37 more
      


      For what I can see in the relevant code is the following:

      1. the ORB$2.evaluate calls ORBClassLoader.getClassLoader.loadClass(...)
      2. ORBClassLoader.getClassLoader() returns the Thread-ContextClassLoader
      3. in ApplicationLifecycle.deploy() the current Thread ContextClassLoader is set to the ClassLoader of the DeploymentContext
      4. the OSGiDeploymentContext internally only uses the Bundles' current ClassLoader or the APIClassLoader


      What we need here would probably be the ConnectorClassLoader as well.

      Also I wonder if implementing the following methods properly would help here:

      • createDeploymentClassLoader
      • createApplicationClassLoader


      Btw., the OSGi application has been deployed using asadmin deploy --type osgi, so not using plain FileInstall.

        Activity

        Hide
        Sanjeeb Sahoo added a comment -

        Ancoron,

        Can you supply a test case?

        Sahoo

        Show
        Sanjeeb Sahoo added a comment - Ancoron, Can you supply a test case? Sahoo
        Hide
        ancoron added a comment -

        I was testing my glassfish-resources extender in a local cluster setup while encountering this.

        Please find the test bundle attached (org.ancoron.osgi.test.bundles-glassfish-resources.jar).

        Also you'll need the actual extender from here:
        https://oss.sonatype.org/content/repositories/releases/org/ancoron/glassfish/extender/resources/1.0.4/

        ...and a small library from here:
        https://oss.sonatype.org/content/repositories/releases/org/ancoron/glassfish/lib/resources/1.0.4/

        Show
        ancoron added a comment - I was testing my glassfish-resources extender in a local cluster setup while encountering this. Please find the test bundle attached ( org.ancoron.osgi.test.bundles-glassfish-resources.jar ). Also you'll need the actual extender from here: https://oss.sonatype.org/content/repositories/releases/org/ancoron/glassfish/extender/resources/1.0.4/ ...and a small library from here: https://oss.sonatype.org/content/repositories/releases/org/ancoron/glassfish/lib/resources/1.0.4/
        Hide
        jkreutzfeld added a comment -

        We're encountering the same problem in our project. We have deployed various OSGi bundles (via the Felix console), of which one contains EJBs and a persistence.xml.
        Everything's fine until we deploy a JasperReports WAR via the Glassfish Admin Console (which i think is the same as using asadmin deploy). After that, a GlassFish restart leads to the exact same stacktrace described by ancoron.

        Our company is very interested in this bug getting fixed. It's a serious issue for maintenance in production environment.

        Show
        jkreutzfeld added a comment - We're encountering the same problem in our project. We have deployed various OSGi bundles (via the Felix console), of which one contains EJBs and a persistence.xml. Everything's fine until we deploy a JasperReports WAR via the Glassfish Admin Console (which i think is the same as using asadmin deploy ). After that, a GlassFish restart leads to the exact same stacktrace described by ancoron. Our company is very interested in this bug getting fixed. It's a serious issue for maintenance in production environment.
        Hide
        Jochen Kraushaar added a comment -

        At least in our case, the problem is a GlassFish / Apache Felix concurrency issue:

        During start up of OSGi bundles containing EJBs, class org.glassfish.enterprise.iiop.impl.GlassFishORBManager tries to start module glassfish-corba-orb in method initOrb(). The start call of the bundle is delegated to the Felix framework. If - at the same time - the FelixStartLevel thread is active, starting of the glassfish-corba-orb bundle is delegated to the FelixStartLevel thread and the start method returns without actually starting the module. The original thread proceeds and then throws the ClassNotFoundException, as the glassfish-corba-orb module is not started, yet.

        In my opinion this bug can be solved by waiting for the glassfish-corba-orb module to be started in GlassFishORBManager. The thread should only proceed, if the bundle is definitely started.

        As a workaround the glassfish-corba-orb module can be started manually e.g. by a BundleActivator. The BundleActivator has to be started before the EJB bundles become started, e.g. by setting a start level of 1.

        Example of such a BundleActivator:

        public class PatchBundleActivator implements BundleActivator
        {
           @Override
           public void start(BundleContext context) throws Exception
           {
              Bundle[] bundles = context.getBundles();
              for (Bundle bundle : bundles) {
                 if ("glassfish-corba-orb".equals(bundle.getSymbolicName())) {
                    bundle.start();
                 }
              }
           }
        
           @Override
           public void stop(BundleContext context) throws Exception
           {
           }
        }
        
        Show
        Jochen Kraushaar added a comment - At least in our case, the problem is a GlassFish / Apache Felix concurrency issue: During start up of OSGi bundles containing EJBs, class org.glassfish.enterprise.iiop.impl.GlassFishORBManager tries to start module glassfish-corba-orb in method initOrb(). The start call of the bundle is delegated to the Felix framework. If - at the same time - the FelixStartLevel thread is active, starting of the glassfish-corba-orb bundle is delegated to the FelixStartLevel thread and the start method returns without actually starting the module. The original thread proceeds and then throws the ClassNotFoundException, as the glassfish-corba-orb module is not started, yet. In my opinion this bug can be solved by waiting for the glassfish-corba-orb module to be started in GlassFishORBManager. The thread should only proceed, if the bundle is definitely started. As a workaround the glassfish-corba-orb module can be started manually e.g. by a BundleActivator. The BundleActivator has to be started before the EJB bundles become started, e.g. by setting a start level of 1. Example of such a BundleActivator: public class PatchBundleActivator implements BundleActivator { @Override public void start(BundleContext context) throws Exception { Bundle[] bundles = context.getBundles(); for (Bundle bundle : bundles) { if ( "glassfish-corba-orb" .equals(bundle.getSymbolicName())) { bundle.start(); } } } @Override public void stop(BundleContext context) throws Exception { } }

          People

          • Assignee:
            Sanjeeb Sahoo
            Reporter:
            ancoron
          • Votes:
            3 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated: