glassfish
  1. glassfish
  2. GLASSFISH-20383

[regression] NCDFE during shutdown of glassfish

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 4.0_b82_EE7MS7
    • Fix Version/s: 4.0
    • Component/s: hk2
    • Labels:
      None

      Description

      To reproduce:
      java -jar modules/glassfish.jar
      Allow it to start. You will see a message like following in the console:

      Registered com.sun.enterprise.glassfish.bootstrap.osgi.EmbeddedOSGiGlassFishImpl@747e2f as OSGi service registration:

      Now 'Ctrl C' or Kill it. You will notice the following message in the console:
      Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "GlassFish Shutdown Hook"

      This didn't use to happen.

        Activity

        Hide
        jwells added a comment -

        Here is a better stack trace:

        java.lang.NoClassDefFoundError: org/glassfish/hk2/runlevel/internal/CurrentTaskFuture$DownAllTheWay
        at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture.<init>(CurrentTaskFuture.java:106)
        at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.proceedTo(AsyncRunLevelContext.java:279)
        at org.glassfish.hk2.runlevel.internal.RunLevelControllerImpl.proceedTo(RunLevelControllerImpl.java:66)
        at com.sun.enterprise.v3.server.AppServerStartup.proceedTo(AppServerStartup.java:555)
        at com.sun.enterprise.v3.server.AppServerStartup.stop(AppServerStartup.java:508)
        at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.stop(GlassFishImpl.java:88)
        at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.dispose(GlassFishImpl.java:97)
        at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.dispose(GlassFishDecorator.java:73)
        at com.sun.enterprise.glassfish.bootstrap.osgi.EmbeddedOSGiGlassFishRuntime.shutdown(EmbeddedOSGiGlassFishRuntime.java:112)
        at com.sun.enterprise.glassfish.bootstrap.GlassFishRuntimeDecorator.shutdown(GlassFishRuntimeDecorator.java:63)
        at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishRuntime.shutdown(OSGiGlassFishRuntime.java:82)
        at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher$1.run(GlassFishMain.java:212)
        Caused by: java.lang.ClassNotFoundException: Unable to load class 'org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$DownAllTheWay' because the bundle wiring for org.glassfish.hk2.runlevel is no longer valid.
        at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1494)
        at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75)
        at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1955)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
        ... 12 more

        Show
        jwells added a comment - Here is a better stack trace: java.lang.NoClassDefFoundError: org/glassfish/hk2/runlevel/internal/CurrentTaskFuture$DownAllTheWay at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture.<init>(CurrentTaskFuture.java:106) at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.proceedTo(AsyncRunLevelContext.java:279) at org.glassfish.hk2.runlevel.internal.RunLevelControllerImpl.proceedTo(RunLevelControllerImpl.java:66) at com.sun.enterprise.v3.server.AppServerStartup.proceedTo(AppServerStartup.java:555) at com.sun.enterprise.v3.server.AppServerStartup.stop(AppServerStartup.java:508) at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.stop(GlassFishImpl.java:88) at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.dispose(GlassFishImpl.java:97) at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.dispose(GlassFishDecorator.java:73) at com.sun.enterprise.glassfish.bootstrap.osgi.EmbeddedOSGiGlassFishRuntime.shutdown(EmbeddedOSGiGlassFishRuntime.java:112) at com.sun.enterprise.glassfish.bootstrap.GlassFishRuntimeDecorator.shutdown(GlassFishRuntimeDecorator.java:63) at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishRuntime.shutdown(OSGiGlassFishRuntime.java:82) at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher$1.run(GlassFishMain.java:212) Caused by: java.lang.ClassNotFoundException: Unable to load class 'org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$DownAllTheWay' because the bundle wiring for org.glassfish.hk2.runlevel is no longer valid. at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1494) at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75) at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1955) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) ... 12 more
        Hide
        jwells added a comment -

        I guess I could cause this to stop happening by forcing the loading of the DownAllTheWay class early on. But that seems... stupid...

        Show
        jwells added a comment - I guess I could cause this to stop happening by forcing the loading of the DownAllTheWay class early on. But that seems... stupid...
        Hide
        jwells added a comment -

        The problem is in OSGiGlassFishRuntime:

        public void shutdown() throws GlassFishException {
        if (framework == null)

        { return; // already shutdown }

        try

        { framework.stop(); framework.waitForStop(0); }

        catch (InterruptedException ex)

        { throw new GlassFishException(ex); } catch (BundleException ex) { throw new GlassFishException(ex); }

        super.shutdown();
        framework = null; // guard against repeated calls.
        }

        Notice that the shutdown of the framework happens before the super.shutdown().

        But that means that if any of the shutdown work needs to load any classes, they will fail (as we see in this bug).

        So I am wondering if this ordering is on purpose, and what the effect would be on changing the ordering to do the super.shutdown first, and the framework shutdown last.

        I will try it now, and verify that it fixes this problem and report back in this bug.

        Show
        jwells added a comment - The problem is in OSGiGlassFishRuntime: public void shutdown() throws GlassFishException { if (framework == null) { return; // already shutdown } try { framework.stop(); framework.waitForStop(0); } catch (InterruptedException ex) { throw new GlassFishException(ex); } catch (BundleException ex) { throw new GlassFishException(ex); } super.shutdown(); framework = null; // guard against repeated calls. } Notice that the shutdown of the framework happens before the super.shutdown(). But that means that if any of the shutdown work needs to load any classes, they will fail (as we see in this bug). So I am wondering if this ordering is on purpose, and what the effect would be on changing the ordering to do the super.shutdown first, and the framework shutdown last. I will try it now, and verify that it fixes this problem and report back in this bug.
        Hide
        jwells added a comment -

        In fact, when I change the order this nice thing happens after ^C:

        ^C
        Completed shutdown of Log manager service
        Completed shutdown of GlassFish runtime

        Show
        jwells added a comment - In fact, when I change the order this nice thing happens after ^C: ^C Completed shutdown of Log manager service Completed shutdown of GlassFish runtime
        Hide
        jwells added a comment -

        Here is the diff that fixes this bug:

        Index: nucleus/core/bootstrap/src/main/java/com/sun/enterprise/glassfish/bootstrap/osgi/OSGiGlassFishRuntime.java
        ===================================================================
        — nucleus/core/bootstrap/src/main/java/com/sun/enterprise/glassfish/bootstrap/osgi/OSGiGlassFishRuntime.java (revision 61615)
        +++ nucleus/core/bootstrap/src/main/java/com/sun/enterprise/glassfish/bootstrap/osgi/OSGiGlassFishRuntime.java (working copy)
        @@ -72,6 +72,8 @@
        return; // already shutdown
        }
        try

        { + super.shutdown(); + framework.stop(); framework.waitForStop(0); }

        catch (InterruptedException ex)

        { @@ -79,8 +81,9 @@ }

        catch (BundleException ex)

        { throw new GlassFishException(ex); }
        • super.shutdown();
        • framework = null; // guard against repeated calls.
          + finally { + framework = null; // guard against repeated calls. + }

          }

        @Override

        Show
        jwells added a comment - Here is the diff that fixes this bug: Index: nucleus/core/bootstrap/src/main/java/com/sun/enterprise/glassfish/bootstrap/osgi/OSGiGlassFishRuntime.java =================================================================== — nucleus/core/bootstrap/src/main/java/com/sun/enterprise/glassfish/bootstrap/osgi/OSGiGlassFishRuntime.java (revision 61615) +++ nucleus/core/bootstrap/src/main/java/com/sun/enterprise/glassfish/bootstrap/osgi/OSGiGlassFishRuntime.java (working copy) @@ -72,6 +72,8 @@ return; // already shutdown } try { + super.shutdown(); + framework.stop(); framework.waitForStop(0); } catch (InterruptedException ex) { @@ -79,8 +81,9 @@ } catch (BundleException ex) { throw new GlassFishException(ex); } super.shutdown(); framework = null; // guard against repeated calls. + finally { + framework = null; // guard against repeated calls. + } } @Override
        Hide
        jwells added a comment -

        What is the impact on the customer of the bug?

        The server will shutdown properly when hit with a kill (15) signal

        What is the cost/risk of fixing the bug?

        I'm not sure of the cost, and I'd rate the fix as being medium to low risk

        Is there an impact on documentation or message strings?

        No

        Which tests should QA (re)run to verify the fix did not destabilize GlassFish?

        EJB, Admin

        Which is the targeted build of 4.0 for this fix?

        b88

        If this an integration of a new version of a component from another project,
        what are the changes that are being brought in? This might be list of
        Jira issues from that project or a list of revision messages.

        n/a

        Show
        jwells added a comment - What is the impact on the customer of the bug? The server will shutdown properly when hit with a kill (15) signal What is the cost/risk of fixing the bug? I'm not sure of the cost, and I'd rate the fix as being medium to low risk Is there an impact on documentation or message strings? No Which tests should QA (re)run to verify the fix did not destabilize GlassFish? EJB, Admin Which is the targeted build of 4.0 for this fix? b88 If this an integration of a new version of a component from another project, what are the changes that are being brought in? This might be list of Jira issues from that project or a list of revision messages. n/a
        Hide
        jwells added a comment -

        Fixed at change 61642

        Show
        jwells added a comment - Fixed at change 61642

          People

          • Assignee:
            jwells
            Reporter:
            Sanjeeb Sahoo
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: