[GLASSFISH-18953] Unable to start Cluster Instances when using JDK-7 on RHEL 5 Created: 27/Jul/12  Updated: 13/May/13

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1.2
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: jp2011 Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Oracle Glassfish 3.1.2.2
RHEL 5
JDK 7u4


Tags: JDK-7, JDK7, RHEL

 Description   

After upgrading from GF 3.1.2 to 3.1.2.2 on DAS, removing cluster instances and recreating them, the instances will not start using JDK 7. The following exception is seen:

<snip>

[#|2012-07-27T09:04:36.428-0400|WARNING|oracle-glassfish3.1.2|javax.enterprise.system.tools.admin.com.sun.enterprise.v3.admin.cluster|_ThreadID=1570;_ThreadName=Thread-2;|Could not start instance Instance1 on node QA-Node-1 (btsqa01dfw.bot.testnet.rim.net).: Command ' /home/glassfish/glassfish3/glassfish/bin/asadmin --_auxinput - --interactive=false start-local-instance --node QA-Node-1 --sync normal Instance1' failed on node QA-Node-1 (<removed>): Waiting for Instance1 to start ......Error starting instance Instance1.
The server exited prematurely with exit code 1.
Before it died, it produced the following output:

Launching GlassFish on Felix platform
Completed shutdown of GlassFish runtime
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97)
at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55)
Caused by: java.lang.RuntimeException: javax.management.MBeanRegistrationException: Exception thrown in preRegister method
at java.lang.management.ManagementFactory.addMXBean(ManagementFactory.java:824)
at java.lang.management.ManagementFactory.getPlatformMBeanServer(ManagementFactory.java:481)
at org.glassfish.admin.monitor.MonitoringBootstrap.setStatsProviderManagerDelegate(MonitoringBootstrap.java:227)
at org.glassfish.admin.monitor.MonitoringBootstrap.enableMonitoringForProbeProviders(MonitoringBootstrap.java:634)
at org.glassfish.admin.monitor.MonitoringBootstrap.postConstruct(MonitoringBootstrap.java:176)
at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:131)
at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:91)
at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:82)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67)
at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:139)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:78)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:229)
at com.sun.enterprise.v3.server.AppServerStartup.doStart(AppServerStartup.java:145)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:136)
at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79)
at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63)
at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishImpl.start(OSGiGlassFishImpl.java:69)
at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117)
... 6 more
Caused by: javax.management.MBeanRegistrationException: Exception thrown in preRegister method
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.throwMBeanRegistrationException(DefaultMBeanServerInterceptor.java:993)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.preRegister(DefaultMBeanServerInterceptor.java:1009)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:919)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:512)
at com.sun.enterprise.v3.admin.DynamicInterceptor.registerMBean(DynamicInterceptor.java:472)
at java.lang.management.ManagementFactory$2.run(ManagementFactory.java:819)
at java.lang.management.ManagementFactory$2.run(ManagementFactory.java:815)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.management.ManagementFactory.addMXBean(ManagementFactory.java:815)
... 23 more
Caused by: javax.management.InstanceAlreadyExistsException: MXBean already registered with name java.lang:type=GarbageCollector,name=PS Scavenge
at com.sun.jmx.mbeanserver.MXBeanLookup.addReference(MXBeanLookup.java:151)
at com.sun.jmx.mbeanserver.MXBeanSupport.register(MXBeanSupport.java:160)
at javax.management.StandardMBean.preRegister(StandardMBean.java:1075)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.preRegister(DefaultMBeanServerInterceptor.java:1007)
... 32 more

Command start-local-instance failed.: To complete this operation run the following command locally on host <removed> from the GlassFish install location /home/glassfish/glassfish3:

bin/asadmin start-local-instance --node QA-Node-1 --sync normal Instance1|#]

[#|2012-07-27T09:04:36.434-0400|SEVERE|oracle-glassfish3.1.2|org.glassfish.admingui|_ThreadID=1567;_ThreadName=Thread-2;|RestResponse.getResponse() gives FAILURE. endpoint = 'https://localhost:4848/management/domain/servers/server/Instance1/start-instance'; attrs = '{}'|#]

[#|2012-07-27T09:04:36.435-0400|SEVERE|oracle-glassfish3.1.2|org.glassfish.admingui|_ThreadID=1567;_ThreadName=Thread-2;|Error in instanceAction ;
endpoint=https://localhost:4848/management/domain/servers/server/Instance1/start-instance;attrsMap=null|#]

</snip>

After downgrading java to JDK 6, everything works correctly. It should be noted that this behaviour is seen for the following JDK versions: 1.7.0_02, 1.7.0_03, 1.7.0_04, 1.7.0_05



 Comments   
Comment by kneecha [ 09/Aug/12 ]

I am having the same issue using 1.7.0_04-b20. The stack trace is basically the same except the GarbageCollector, name is different:

Caused by: javax.management.InstanceAlreadyExistsException: MXBean already registered with name java.lang:type=GarbageCollector,name=ParNew

Comment by skgaju [ 24/Aug/12 ]

workaround remove the jvm-option from all server/clusters configs
NOT from DAS config
-Djavax.management.builder.initial=com.sun.enterprise.v3.admin.AppServerMBeanServerBuilder

Fix for this issue

bash-2.05b# svn diff src/main/java/com/sun/enterprise/admin/servermgmt/V2ToV3ConfigUpgrade.java
Index: src/main/java/com/sun/enterprise/admin/servermgmt/V2ToV3ConfigUpgrade.java
===================================================================
— src/main/java/com/sun/enterprise/admin/servermgmt/V2ToV3ConfigUpgrade.java (revision 5906)
+++ src/main/java/com/sun/enterprise/admin/servermgmt/V2ToV3ConfigUpgrade.java (working copy)
@@ -152,6 +152,7 @@
"-XX:LogFile",
};

+
// these are added to all configs
private static final String[] ADD_LIST = new String[] {
"-XX:+UnlockDiagnosticVMOptions",
@@ -169,14 +170,14 @@
"-Dfelix.fileinstall.bundles.startTransient=true",
"-Dfelix.fileinstall.disableConfigSave=false",
"-Dfelix.fileinstall.log.level=2",

  • "-Djavax.management.builder.initial=com.sun.enterprise.v3.admin.AppServerMBeanServerBuilder",
    "-Dorg.glassfish.web.rfc2109_cookie_names_enforced=false",
    "-Djava.ext.dirs=$ {com.sun.aas.javaRoot}/lib/ext${path.separator}${com.sun.aas.javaRoot}

    /jre/lib/ext$

    {path.separator}

    $

    {com.sun.aas.instanceRoot}

    /lib/ext",
    };

// these are added to DAS only
private static final String[] ADD_LIST_DAS = new String[]

{ - "-Dosgi.shell.telnet.port=6666" + "-Dosgi.shell.telnet.port=6666", + "-Djavax.management.builder.initial=com.sun.enterprise.v3.admin.AppServerMBeanServerBuilder" }

;

Comment by manuel_b [ 13/May/13 ]

I am also encountering this issue after upgrading our cluster nodes from Java 6 to Java 7:

..
Caused by: javax.management.InstanceAlreadyExistsException: MXBean already registered with name java.lang:type=GarbageCollector,name=Copy
at com.sun.jmx.mbeanserver.MXBeanLookup.addReference(MXBeanLookup.java:151)
at com.sun.jmx.mbeanserver.MXBeanSupport.register(MXBeanSupport.java:160)
...

I will remove the mentioned jvm property from the node configuration.

Works! Thanks for the hints here.

Generated at Wed Sep 28 02:56:44 UTC 2016 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.