[GLASSFISH-18953] Unable to start Cluster Instances when using JDK-7 on RHEL 5 Created: 27/Jul/12  Updated: 13/May/13

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1.2
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: jp2011 Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Oracle Glassfish 3.1.2.2
RHEL 5
JDK 7u4


Tags: JDK-7, JDK7, RHEL

 Description   

After upgrading from GF 3.1.2 to 3.1.2.2 on DAS, removing cluster instances and recreating them, the instances will not start using JDK 7. The following exception is seen:

<snip>

[#|2012-07-27T09:04:36.428-0400|WARNING|oracle-glassfish3.1.2|javax.enterprise.system.tools.admin.com.sun.enterprise.v3.admin.cluster|_ThreadID=1570;_ThreadName=Thread-2;|Could not start instance Instance1 on node QA-Node-1 (btsqa01dfw.bot.testnet.rim.net).: Command ' /home/glassfish/glassfish3/glassfish/bin/asadmin --_auxinput - --interactive=false start-local-instance --node QA-Node-1 --sync normal Instance1' failed on node QA-Node-1 (<removed>): Waiting for Instance1 to start ......Error starting instance Instance1.
The server exited prematurely with exit code 1.
Before it died, it produced the following output:

Launching GlassFish on Felix platform
Completed shutdown of GlassFish runtime
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97)
at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55)
Caused by: java.lang.RuntimeException: javax.management.MBeanRegistrationException: Exception thrown in preRegister method
at java.lang.management.ManagementFactory.addMXBean(ManagementFactory.java:824)
at java.lang.management.ManagementFactory.getPlatformMBeanServer(ManagementFactory.java:481)
at org.glassfish.admin.monitor.MonitoringBootstrap.setStatsProviderManagerDelegate(MonitoringBootstrap.java:227)
at org.glassfish.admin.monitor.MonitoringBootstrap.enableMonitoringForProbeProviders(MonitoringBootstrap.java:634)
at org.glassfish.admin.monitor.MonitoringBootstrap.postConstruct(MonitoringBootstrap.java:176)
at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:131)
at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:91)
at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:82)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67)
at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:139)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:78)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:229)
at com.sun.enterprise.v3.server.AppServerStartup.doStart(AppServerStartup.java:145)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:136)
at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79)
at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63)
at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishImpl.start(OSGiGlassFishImpl.java:69)
at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117)
... 6 more
Caused by: javax.management.MBeanRegistrationException: Exception thrown in preRegister method
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.throwMBeanRegistrationException(DefaultMBeanServerInterceptor.java:993)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.preRegister(DefaultMBeanServerInterceptor.java:1009)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:919)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:512)
at com.sun.enterprise.v3.admin.DynamicInterceptor.registerMBean(DynamicInterceptor.java:472)
at java.lang.management.ManagementFactory$2.run(ManagementFactory.java:819)
at java.lang.management.ManagementFactory$2.run(ManagementFactory.java:815)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.management.ManagementFactory.addMXBean(ManagementFactory.java:815)
... 23 more
Caused by: javax.management.InstanceAlreadyExistsException: MXBean already registered with name java.lang:type=GarbageCollector,name=PS Scavenge
at com.sun.jmx.mbeanserver.MXBeanLookup.addReference(MXBeanLookup.java:151)
at com.sun.jmx.mbeanserver.MXBeanSupport.register(MXBeanSupport.java:160)
at javax.management.StandardMBean.preRegister(StandardMBean.java:1075)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.preRegister(DefaultMBeanServerInterceptor.java:1007)
... 32 more

Command start-local-instance failed.: To complete this operation run the following command locally on host <removed> from the GlassFish install location /home/glassfish/glassfish3:

bin/asadmin start-local-instance --node QA-Node-1 --sync normal Instance1|#]

[#|2012-07-27T09:04:36.434-0400|SEVERE|oracle-glassfish3.1.2|org.glassfish.admingui|_ThreadID=1567;_ThreadName=Thread-2;|RestResponse.getResponse() gives FAILURE. endpoint = 'https://localhost:4848/management/domain/servers/server/Instance1/start-instance'; attrs = '{}'|#]

[#|2012-07-27T09:04:36.435-0400|SEVERE|oracle-glassfish3.1.2|org.glassfish.admingui|_ThreadID=1567;_ThreadName=Thread-2;|Error in instanceAction ;
endpoint=https://localhost:4848/management/domain/servers/server/Instance1/start-instance;attrsMap=null|#]

</snip>

After downgrading java to JDK 6, everything works correctly. It should be noted that this behaviour is seen for the following JDK versions: 1.7.0_02, 1.7.0_03, 1.7.0_04, 1.7.0_05



 Comments   
Comment by kneecha [ 09/Aug/12 ]

I am having the same issue using 1.7.0_04-b20. The stack trace is basically the same except the GarbageCollector, name is different:

Caused by: javax.management.InstanceAlreadyExistsException: MXBean already registered with name java.lang:type=GarbageCollector,name=ParNew

Comment by skgaju [ 24/Aug/12 ]

workaround remove the jvm-option from all server/clusters configs
NOT from DAS config
-Djavax.management.builder.initial=com.sun.enterprise.v3.admin.AppServerMBeanServerBuilder

Fix for this issue

bash-2.05b# svn diff src/main/java/com/sun/enterprise/admin/servermgmt/V2ToV3ConfigUpgrade.java
Index: src/main/java/com/sun/enterprise/admin/servermgmt/V2ToV3ConfigUpgrade.java
===================================================================
— src/main/java/com/sun/enterprise/admin/servermgmt/V2ToV3ConfigUpgrade.java (revision 5906)
+++ src/main/java/com/sun/enterprise/admin/servermgmt/V2ToV3ConfigUpgrade.java (working copy)
@@ -152,6 +152,7 @@
"-XX:LogFile",
};

+
// these are added to all configs
private static final String[] ADD_LIST = new String[] {
"-XX:+UnlockDiagnosticVMOptions",
@@ -169,14 +170,14 @@
"-Dfelix.fileinstall.bundles.startTransient=true",
"-Dfelix.fileinstall.disableConfigSave=false",
"-Dfelix.fileinstall.log.level=2",

  • "-Djavax.management.builder.initial=com.sun.enterprise.v3.admin.AppServerMBeanServerBuilder",
    "-Dorg.glassfish.web.rfc2109_cookie_names_enforced=false",
    "-Djava.ext.dirs=$ {com.sun.aas.javaRoot}/lib/ext${path.separator}${com.sun.aas.javaRoot}

    /jre/lib/ext$

    {path.separator}

    $

    {com.sun.aas.instanceRoot}

    /lib/ext",
    };

// these are added to DAS only
private static final String[] ADD_LIST_DAS = new String[]

{ - "-Dosgi.shell.telnet.port=6666" + "-Dosgi.shell.telnet.port=6666", + "-Djavax.management.builder.initial=com.sun.enterprise.v3.admin.AppServerMBeanServerBuilder" }

;

Comment by manuel_b [ 13/May/13 ]

I am also encountering this issue after upgrading our cluster nodes from Java 6 to Java 7:

..
Caused by: javax.management.InstanceAlreadyExistsException: MXBean already registered with name java.lang:type=GarbageCollector,name=Copy
at com.sun.jmx.mbeanserver.MXBeanLookup.addReference(MXBeanLookup.java:151)
at com.sun.jmx.mbeanserver.MXBeanSupport.register(MXBeanSupport.java:160)
...

I will remove the mentioned jvm property from the node configuration.

Works! Thanks for the hints here.





[GLASSFISH-16525] Add Auto Restarting Capability to Platform Services Created: 02/May/11  Updated: 17/Oct/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1
Fix Version/s: future release

Type: Improvement Priority: Blocker
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks GLASSFISH-16311 Improve operating service (OS) integr... Open
Tags: ee7ri_cleanup_deferred

 Description   

Automatic restart of crashed GlassFish servers



 Comments   
Comment by Tom Mueller [ 17/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issues was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-14389] [BLOCKING] asadmin -m * and asadmin --monitor=true * is not working as it mention on the online --help. Created: 03/Nov/10  Updated: 18/Feb/13

Status: Reopened
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: future release

Type: Bug Priority: Critical
Reporter: Homer Yau Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issue Links:
Dependency
depends on GLASSFISH-14461 Extra "server." in monitoring output ... Resolved
Issuezilla Id: 14,389
Tags: added-devtest

 Description   

"asadmin -m *" and "asadmin --monitor=true *" is not working as it mention on
the online --help.

I have created a cluster instance for monitoring test and turn it to "HIGH".

-bash-3.2$ asadmin list -m *
Command list executed successfully.

-bash-3.2$ asadmin list -m=true *
Command list executed successfully.

You could see what online help says, I think it confuse user.
It says the " ./asadmin list * " to see all the monitoring element.

-bash-3.2$ asadmin list --help

User need to know the element start point and need to able to list.
The following command works.

-bash-3.2$ asadmin list -m=true monitoring-server.*
-bash-3.2$ asadmin list -m monitoring-server.*
monitoring-server :
monitoring-server.server
monitoring-server.server.applications
monitoring-server.server.applications.__default-web-module
monitoring-server.server.applications.__default-web-module.server
monitoring-server.server.applications.__default-web-module.server.default
monitoring-server.server.applications.__default-web-module.server.jsp
monitoring-server.server.applications.standalonewebmod1
monitoring-server.server.applications.standalonewebmod1.server
monitoring-server.server.applications.standalonewebmod1.server.default
monitoring-server.server.applications.standalonewebmod1.server.jsp
monitoring-server.server.applications.standalonewebmod1.server.standalonewebmod1_Servlet1
monitoring-server.server.applications.standalonewebmod1.server.standalonewebmod1_Servlet2
monitoring-server.server.applications.standalonewebmod2
monitoring-server.server.applications.standalonewebmod2.server
monitoring-server.server.applications.standalonewebmod2.server.ErrorServlet
monitoring-server.server.applications.standalonewebmod2.server.ReceiveBytes
monitoring-server.server.applications.standalonewebmod2.server.SecureServlet
monitoring-server.server.applications.standalonewebmod2.server.SendBytes
monitoring-server.server.applications.standalonewebmod2.server.default
monitoring-server.server.applications.standalonewebmod2.server.jsp
monitoring-server.server.applications.standalonewebmod2.server.standalonewebmod2_Servlet1
monitoring-server.server.applications.standalonewebmod2.server.standalonewebmod2_Servlet2
monitoring-server.server.applications.webapp1
monitoring-server.server.applications.webapp1.webapp1webmod1\.war
monitoring-server.server.applications.webapp1.webapp1webmod1\.war.server
monitoring-server.server.applications.webapp1.webapp1webmod1\.war.server.default
monitoring-server.server.applications.webapp1.webapp1webmod1\.war.server.jsp
monitoring-server.server.applications.webapp1.webapp1webmod1\.war.server.webapp1webmod1_Servlet1
monitoring-server.server.applications.webapp1.webapp1webmod1\.war.server.webapp1webmod1_Servlet2
monitoring-server.server.applications.webapp1.webapp1webmod2\.war
monitoring-server.server.applications.webapp1.webapp1webmod2\.war.server
monitoring-server.server.applications.webapp1.webapp1webmod2\.war.server.default
monitoring-server.server.applications.webapp1.webapp1webmod2\.war.server.jsp
monitoring-server.server.applications.webapp1.webapp1webmod2\.war.server.webapp1webmod2_Servlet1
monitoring-server.server.applications.webapp1.webapp1webmod2\.war.server.webapp1webmod2_Servlet2
monitoring-server.server.applications.webapp2
monitoring-server.server.applications.webapp2.webapp2webmod1\.war
monitoring-server.server.applications.webapp2.webapp2webmod1\.war.server
monitoring-server.server.applications.webapp2.webapp2webmod1\.war.server.default
monitoring-server.server.applications.webapp2.webapp2webmod1\.war.server.jsp
monitoring-server.server.applications.webapp2.webapp2webmod1\.war.server.webapp2webmod1_Servlet1
monitoring-server.server.applications.webapp2.webapp2webmod1\.war.server.webapp2webmod1_Servlet2
monitoring-server.server.applications.webapp2.webapp2webmod2\.war
monitoring-server.server.applications.webapp2.webapp2webmod2\.war.server
monitoring-server.server.applications.webapp2.webapp2webmod2\.war.server.default
monitoring-server.server.applications.webapp2.webapp2webmod2\.war.server.jsp
monitoring-server.server.applications.webapp2.webapp2webmod2\.war.server.webapp2webmod2_Servlet1
monitoring-server.server.applications.webapp2.webapp2webmod2\.war.server.webapp2webmod2_Servlet2
monitoring-server.server.web
monitoring-server.server.web.jsp
monitoring-server.server.web.request
monitoring-server.server.web.servlet
monitoring-server.server.web.session

Command list executed successfully.



 Comments   
Comment by Nazrul [ 03/Nov/10 ]

This is blocking tests execution for monitoring.

Comment by Byron Nevins [ 03/Nov/10 ]

I'm guessing that it is behaving exactly the same as V3 final release – in
which case it will be a big deal...

Comment by Byron Nevins [ 05/Nov/10 ]

"I have created a cluster instance for monitoring test and turn it to "HIGH"."

What does the above mean? How about simply copying & pasting the actual
commands you ran?

Comment by Byron Nevins [ 05/Nov/10 ]

What did you set to high?

Comment by Byron Nevins [ 05/Nov/10 ]

Output from 3.0.1

C:\glassfishv3\glassfish\bin>asadmin list -m "*"
server
server.applications
server.applications.AuthAdmin
server.applications.AuthAdmin.server
server.applications.AuthAdmin.server.AuthCreationManager
server.applications.AuthAdmin.server.AuthManager
server.applications.AuthAdmin.server.AuthUpdateManager
server.applications.AuthAdmin.server.Registrar
server.applications.AuthAdmin.server.Signup
server.applications.AuthAdmin.server.default
server.applications.AuthAdmin.server.jsp
server.applications.__default-web-module
server.applications.__default-web-module.server
server.applications.__default-web-module.server.default
server.applications.__default-web-module.server.jsp
server.http-service
server.http-service.__asadmin
server.http-service.__asadmin.request
server.http-service.server
server.http-service.server.request
server.jvm
server.jvm.class-loading-system
server.jvm.compilation-system
server.jvm.garbage-collectors
server.jvm.garbage-collectors.Copy
server.jvm.garbage-collectors.MarkSweepCompact
server.jvm.memory
server.jvm.operating-system
server.jvm.runtime
server.jvm.thread-system
server.jvm.thread-system.thread-10
server.jvm.thread-system.thread-12
server.jvm.thread-system.thread-13
server.jvm.thread-system.thread-14
server.jvm.thread-system.thread-15
server.jvm.thread-system.thread-16
server.jvm.thread-system.thread-18
server.jvm.thread-system.thread-2
server.jvm.thread-system.thread-20
server.jvm.thread-system.thread-22
server.jvm.thread-system.thread-23
server.jvm.thread-system.thread-24
server.jvm.thread-system.thread-3
server.jvm.thread-system.thread-30
server.jvm.thread-system.thread-31
server.jvm.thread-system.thread-32
server.jvm.thread-system.thread-33
server.jvm.thread-system.thread-34
server.jvm.thread-system.thread-35
server.jvm.thread-system.thread-36
server.jvm.thread-system.thread-37
server.jvm.thread-system.thread-38
server.jvm.thread-system.thread-39
server.jvm.thread-system.thread-4
server.jvm.thread-system.thread-40
server.jvm.thread-system.thread-41
server.jvm.thread-system.thread-42
server.jvm.thread-system.thread-43
server.jvm.thread-system.thread-44
server.jvm.thread-system.thread-45
server.jvm.thread-system.thread-46
server.jvm.thread-system.thread-47
server.jvm.thread-system.thread-49
server.jvm.thread-system.thread-5
server.jvm.thread-system.thread-9
server.network
server.network.admin-listener
server.network.admin-listener.connection-queue
server.network.admin-listener.file-cache
server.network.admin-listener.keep-alive
server.network.admin-listener.thread-pool
server.network.http-listener-1
server.network.http-listener-1.connection-queue
server.network.http-listener-1.file-cache
server.network.http-listener-1.keep-alive
server.network.http-listener-1.thread-pool
server.network.http-listener-2
server.network.http-listener-2.connection-queue
server.network.http-listener-2.file-cache
server.network.http-listener-2.keep-alive
server.network.http-listener-2.thread-pool
server.security
server.security.web
server.transaction-service
server.web
server.web.jsp
server.web.request
server.web.servlet
server.web.session

Command list executed successfully.

Comment by Byron Nevins [ 05/Nov/10 ]

Homer -
I know what you do NOT want it to do.

What do you WANT it to do?

====================================================

I can see exactly why you are getting the results you documented. Vijay
programmed it three months ago to do this:

Take the arg. and parse out everything before the first dot. If there is no dot
– then take the entire string. That is the server name
Then call that server and run the get command on it...

So say you give these args:

server.* --> name of server == "server"

  • --> name of remote server == "*" !!!

============

So in your case it contacts the server named "", sees that "" is not running
and returns nothing.

=========================

Comment by Byron Nevins [ 05/Nov/10 ]

I get no output from:

asadmin list -m monitoring-server.*

I'm guessing that monitoring-server is the name of a server instance you created?
Why am I guessing?

Comment by Byron Nevins [ 05/Nov/10 ]

I have no instructions on WHAT EXACTLY it should do.
I will implement solution #3 if I don't hear back soon...

asadmin list -m "*"

to do

What it does right now is very brain-dead. It blindly assumes that "*" is the
name of a server instance. So obviously that should change. But change to what?

(1) Assume the user wants a list of everything for DAS and every running server
instance. Call every instance and stick all the outputs into one big list

(2) Fail with an error saying something like "You must specify a server"

(3) *Assume* that if there is no server name that the user wants output from
DAS.

I like (3) – it knits together with 3.0.1 output the best.

Comment by Byron Nevins [ 05/Nov/10 ]

Here is how V2 handles 'get -m "*"'

case 1 – only DAS is running

C:\glassfish\bin>asadmin list -m "*"
server
server.applications
server.applications.WSTXServices
server.applications.__JWSappclients
server.applications.__JWSappclients.sys\.war
server.applications.__default-web-module
server.applications.adminapp
server.applications.admingui
server.connector-service
server.http-service
server.http-service.server
server.jms-service
server.jvm
server.orb
server.orb.connection-managers
server.resources
server.thread-pools

case 2: DAS and i1 are running

C:\glassfish\bin>asadmin list -m "*"
i1
i1.applications
i1.applications.WSTXServices
i1.applications.__JWSappclients
i1.applications.__JWSappclients.sys\.war
i1.applications.__default-web-module
i1.connector-service
i1.http-service
i1.http-service.server
i1.jms-service
i1.jvm
i1.orb
i1.orb.connection-managers
i1.resources
i1.thread-pools
server
server.applications
server.applications.WSTXServices
server.applications.__JWSappclients
server.applications.__JWSappclients.sys\.war
server.applications.__default-web-module
server.applications.adminapp
server.applications.admingui
server.connector-service
server.http-service
server.http-service.server
server.jms-service
server.jvm
server.orb
server.orb.connection-managers
server.resources
server.thread-pools

Comment by Byron Nevins [ 07/Nov/10 ]

Notice how the output is:

monitoring-server.server.applications

In V2 it would have been:

monitoring-server.applications

    • blocking bug created for this – 14461
Comment by Byron Nevins [ 07/Nov/10 ]

There used to be 5 lines of code figuring out what part of the "pattern" is
really the regular expression pattern and what part is the target. And handling
defaults, etc.

Now there is ~~ 300 lines of code doing this. It needs massive, but rapid,
testing by Homer informally at a console. E.g. here are a bunch of commands
that should all do what you would hope they would do

You have a cluster c1 with 2 instances c1i1, c1i2 and 2 stand-alone instance i1,i2

asadmin get -m "*" – get EVERYTHING from EVERY instance and DAS
asadmin get -m "server.*" - get everything from DAS
asadmin get -m "c1.*" get everything from c1i1 and c1i2
asadmin get -m "c1i1.*" get everything from c1i1
asadmin get -m "c1i1.server.*" exactly like above. ignore the "server."
asadmin get -m "." – just like ""

And on and on and on – you'll think of more!
asadmin get -m "
asadmin get -m "
asadmin get -m "

Comment by Byron Nevins [ 07/Nov/10 ]

There are 2 parts to this bug – get and list
get is now fixed.
list is next to be fixed.

Homer – get is ready for your testing!

C:\gf\v3\core\kernel>svn commit -F ..\qqq
Sending kernel\src\main\java\com\sun\enterprise\v3\admin\GetCommand.java
Sending
kernel\src\main\java\com\sun\enterprise\v3\admin\LocalStrings.properties
Adding
kernel\src\main\java\com\sun\enterprise\v3\admin\MonitoringReporter.java
Transmitting file data ...
Committed revision 42545.

Comment by Byron Nevins [ 10/Nov/10 ]

List is done now too.

Finis!

Comment by Tom Mueller [ 03/May/12 ]

Reopening this issue because the admin devtests are failing (sometimes) when the test that was written for this issue is enabled.

It appears that this is related to retrieving monitoring data from remote instances.





[GLASSFISH-17983] Automate DCOM setup. Created: 13/Dec/11  Updated: 14/Dec/11

Status: Open
Project: glassfish
Component/s: distributed management
Affects Version/s: 3.1.2_b13
Fix Version/s: None

Type: Improvement Priority: Critical
Reporter: easarina Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3_1_2_review

 Description   

Current DCOM instruction includes a step that requires manual windows registry editing. I don't think, that it would be good to recommend customers to edit registry manually. So, it looks for me, that this step has to be automated.



 Comments   
Comment by sb110099 [ 14/Dec/11 ]

Upgrading the bug to P2, as it needs some evaluation and attention for 3.1.2 .
This manual step for DCOM support will need to be automated for customers from usability perspective.

-Sudipa





[GLASSFISH-18084] das.properties needs work Created: 23/Dec/11  Updated: 18/Feb/13

Status: Open
Project: glassfish
Component/s: distributed management
Affects Version/s: 3.1.2_b15, 4.0_b16
Fix Version/s: future release

Type: Bug Priority: Critical
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks GLASSFISH-18094 Need Status if _create-instance-files... Resolved
Tags: 3_1_2-exclude

 Description   

Say I have this setup:

DAS is running on my laptop which is connected via VPN to a huge corporation. Let's say Oracle. My official hostname is "laptop"
The remote computer that will host instances is sitting at the Corporation behind the firewall.

Now I create a node and an instance on the remote computer using SSH or DCOM. That creates a file called

das.properties

Inside das.properties is the information to call back to DAS. In my case here the hostname is laptop. There are three problems:

(1) the hostname "laptop" is useless. There is NO WAY the remote machine can find its way to my laptop with that name. It would have to have an IP address.

(2) There was NO HANDSHAKE when the instance was created! The command should have failed. The user has no idea that there is no way for the remote to call DAS back ever.

(3) this also happens across domains. E.g. if the 2 machines have these names that are in DNS – it still won't work:
somehost.in.oracle.com
another.us.oracle.com

Why? The domain gets chopped off. If 'another' is the remote it will look for DAS at
somehost.us.oracle.com

(4) I ran it with a secure DAS – yet isSecure is set to false in DAS.properties.
(5) The protocol is set http. Shouldn't it be https?

WORKAROUND:
After das.properties is created, hand-edit the hostname to something that the remote machine can access.



 Comments   
Comment by kshitiz_saxena [ 04/Jan/12 ]

Instead of manual edit, set address for admin-listener in DAS :
asadmin set configs.config.server-config.network-config.network-listeners.network-listener.admin-listener.address=<YOUR IP ADDRESS>

This works.

Comment by Joe Di Pol [ 30/Jan/12 ]

Not a 3.1.2 stopper





[GLASSFISH-18486] create-service does not work on 64-bit Windows Created: 08/Mar/12  Updated: 08/Mar/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1.2
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

winsw.exe doesn't seem to work properly on 64-bit Windows.

Recommended solution - build a 64bit version of winsw.exe






[GLASSFISH-18451] install-node-dcom does not function Created: 05/Mar/12  Updated: 26/Jun/12

Status: Open
Project: glassfish
Component/s: distributed management
Affects Version/s: 3.1.2_b23
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: jp2011 Assignee: Byron Nevins
Resolution: Unresolved Votes: 5
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows server 2008 R2 Sp1
Glassfish 3.1.2 Release


Tags: dcom

 Description   

I initially tried using a windows domain account to install the node and that didn't work as per GLASSFISH-18327. That issue was also incorrectly marked as resolved. Network captures show that the release version of Glassfish 3.1.2 still does not use the domain account, but attempts to use the local account.

After giving up with this, I created a new local account on the remote machine called glassfish. Granted full access to the 2 required registry keys and added the account to the administrators group. Attempting to install the remote node using this account still fails with the following message:

Successfully verified that the host, hostname, is not the local machine as required. Successfully resolved host name to: hostname/10.65.30.xxx Successfully connected to DCOM Port at port 135 on host hostname. Successfully connected to NetBIOS Session Service at port 139 on host hostname. Successfully connected to Windows Shares at port 445 on host hostname. The remote file, C: doesn't exist on hostname: Access is denied.

I performed a network capture and can tell you the following:

1. The user account is successfully authenticated with STATUS_SUCCESS (0x00000000)
2. SMB is attempting to access \\hostname\C$ no matter what I set the remote test directory to.
3. NT Status: STATUS_FS_DRIVER_REQUIRED (0xc000019c) is returned from the remote host but I suspect this is normal and used for dynamic library loading for the file system.

4. NT Status: STATUS_ACCESS_DENIED (0xc0000022) is returned on attempting to connect to \\hostname\C$

The documentation does not state any other prerequisite or permissions that need to be setup for this to function. What is missing?



 Comments   
Comment by Byron Nevins [ 06/Mar/12 ]

what is the exact command you're running?

Comment by jp2011 [ 06/Mar/12 ]

To make things even simpler, it is reproducible by the validate-dcom command alone.

Password file contains the following line: AS_ADMIN_WINDOWSPASSWORD=$

{ALIAS=glassfish-alias}

I have setup the alias already in asadmin as per the documentation.

c:\glassfish3\bin>asadmin --passwordfile passwordfile.txt validate-dcom -w glassfish remotehost
remote failure:
Successfully verified that the host, remotehost, is not the local machine as required.
Successfully resolved host name to: remotehost/10.65.30.187
Successfully connected to DCOM Port at port 135 on host remotehost.
Successfully connected to NetBIOS Session Service at port 139 on host remotehost
nc.
Successfully connected to Windows Shares at port 445 on host remotehost.
The remote file, C: doesn't exist on remotehost: Access is denied.

Command validate-dcom failed.

I can speak to the network capture I took as well, but that would be easier offline to this web portal.

Comment by Byron Nevins [ 06/Mar/12 ]

Can you access the c$ share from another computer – say

net use X: \\other\c$

?

Comment by Byron Nevins [ 06/Mar/12 ]

Please make sure theses items are setup correctly, especially the third one:

1. Server service is in the started state and is set to start automatically.
2. Remote Registry service is also in the started state and is set to start automatically.
3. Set the Local Policy for Network Access:Control Panel" > "Administrative Tools" -> "Local Security Policy"> "Local Policies" -> "Security Options" -> "Network Access: Sharing security model for local accounts" Make sure it is set to Classic

Comment by ljnelson [ 28/Mar/12 ]

I have exactly the same problem.

I installed and ran setup-local-dcom on the remote machine as an administrator. It claimed it ran successfully.

Then I made sure that your steps 1-3 above were taken. I had to manually start the remote registry service.

My remote machine is running Windows 7 Professional on a 64-bit machine with all updates installed.

Here is my command and output:

ljnelson$ asadmin --passwordfile ~/.glassfish.passwords --port=9048 validate-dcom --windowsuser lnelson --windowsdomain jenzabar --remotetestdir 'C:\crap' --verbose true 10.63.4.42
remote failure: 
Successfully verified that the host, 10.63.4.42, is not the local machine as required.
Successfully resolved host name to: /10.63.4.42
Successfully connected to DCOM Port at port 135 on host 10.63.4.42.
Successfully connected to NetBIOS Session Service at port 139 on host 10.63.4.42.
Successfully connected to Windows Shares at port 445 on host 10.63.4.42.
The remote file, C:\crap doesn't exist on 10.63.4.42 : The parameter is incorrect.

Command validate-dcom failed.

C:\crap is a directory present on the remote machine. I haven't set it up to be shared in any way, but I haven't done anything else to it, either. Any path supplied to --remotetestdir is considered to not exist. I've tried moving slashes around and doubling up backslashes in case it's a path issue; it's not.

Hope this data point helps.

Comment by lb54 [ 11/Apr/12 ]

Hi.
I have also this issue:
Win 2003 SP2 (Domain Admin Server, GF 3.1.1, updated to 3.1.2)
Win 2008 Server R2 Enterprise SP1 (node, formerly connected through SSH via cygwin)
User is authorized for both machines. DCOM is planned to replace the SSH-communication.

Message is from Web Console is:
Successfully verified that the host, myserver.host.xx, is not the local machine as required. Successfully resolved host name to: myserver.host.xx/<IP-Address> Successfully connected to DCOM Port at port 135 on host myserver.host.xx. Successfully connected to NetBIOS Session Service at port 139 on host gibson-10.tecis.hh. Successfully connected to Windows Shares at port 445 on host myserver.host.xx. The remote file, C: doesn't exist on myserver.host.xx : Logon failure: unknown user name or bad password.

The CLI also fails with:
remote failure: Command install-node-dcom failed.

com.sun.enterprise.util.cluster.windows.process.WindowsException: Logon failure: unknown user name or bad password.
Command create-node-dcom failed.

Is there a way to "workaround" this or do I have to wait for an update?

Comment by jp2011 [ 11/Apr/12 ]

There has been no fix for this because the cause is still unknown to Oracle. The workaround is do not use DCOM. We have personally abandoned Windows as a platform for production/QA in favour of RHEL 5 Linux distro. SSH is built in, and the cluster runs a lot faster with less overhead. The downside is that you have to learn Linux commands. But really, is this that bad?

Comment by lb54 [ 13/Apr/12 ]

I agree with you.
BUT: Telling my company to use Linux Servers instead of Windows will not work, they don't want to hear that.
Using SSH Nodes on Windows System with cygwin seems to be an alternative. But I used Glassfish 3.1.1 with ssh (cygwin) already, the communication seems to be not very stable (long running startup processes and long loading "Clusters" page with the Web Console).

@Byron: Is there a plan for this bugfix so far?

Comment by mr_daemon [ 16/Apr/12 ]

I did some incredibly tedious debugging and was able to get it to work:

For the validate-dcom test to pass, since it seems to ignore the parameter for the test directory entirely and always use C:\ regardless, you must disable the new (vista+) policy that prevents users from elevating their privileges over the network by navigatinig to

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System

and creating a new DWORD named LocalAccountTokenFilterPolicy of 1. This will allow the delete-me.bat file to be created there.

However this then breaks again:

PS D:\private> d:\glassfish3\bin\asadmin.bat --passwordfile dcom-pw.txt validate-dcom -w glassfish -v=true qlsvrnode2
remote failure:
Successfully verified that the host, qlsvrnode2, is not the local machine as required.
Successfully resolved host name to: qlsvrnode2/192.168.9.11
Successfully connected to DCOM Port at port 135 on host qlsvrnode2.
Successfully connected to NetBIOS Session Service at port 139 on host qlsvrnode2.
Successfully connected to Windows Shares at port 445 on host qlsvrnode2.
Successfully accessed C: on qlsvrnode2 using DCOM.
Successfully wrote delete_me.bat to C: on qlsvrnode2 using DCOM.
Could not connect to WMI (Windows Management Interface) on qlsvrnode2. : Error setting up remote connection to WMI

This is not mentionned at all in the documentation, but turns out you also need to change ownership and set permissions to the following registry key, in addition to the ones already listed:

HKEY_CLASSES_ROOT\CLSID\{76A64158-CB41-11D1-8B02-00600806D9B6}

Once this is accomplished, everything works as advertised.

I am not fond of the security implications but at least it works and is at least more reliable than Cygwin+sshd.

Comment by Byron Nevins [ 19/Apr/12 ]

Thanks for the excellent comments and work everyone. I'll try and address this problem soon.

Comment by lb54 [ 26/Apr/12 ]

Hi Byron.
Are there any plans to release this fix so far? Or is the "hack" described above the official solution?

Thanks for info.

Best wishes.

Basti

Comment by lb54 [ 16/May/12 ]

Hi there.
It seems that no one is working on this ticket right now.
Is there a chance to get a fix for this in the near future?
Unfortunatly the "quick fix" described above does not work for me, so I need another workaround or this bug fixed.

Can anyone help me?

Thanks.

Basti

Comment by mtobler [ 25/Jun/12 ]

I have not been able to get this to work on a set of 2008 R2 Servers
which I am trying to cluster. Unfortunately I am unable to get the ssh
functionality to work as well which leaves me with no clustering
capability and wondering why we used Glassfish.
Is anyone going to work on this anytime soon?

I added the following to 18327 but am adding it here as requested:
asadmin> validate-dcom --passwordfile do-not-delete gf01
remote failure:
Successfully verified that the host, gf01, is not the local machine as required.
Successfully resolved host name to: gf01/172.18.11.169
Successfully connected to DCOM Port at port 135 on host gf01.
Successfully connected to NetBIOS Session Service at port 139 on host gf01.
Successfully connected to Windows Shares at port 445 on host gf01.
The remote file, C: doesn't exist on gf01 : Logon failure: unknown user name or bad password.

I am using a domain and the user is a domain user.

I have gone through every document I can find on this issue and have verified all settings/registry keys/etc are correct. I have tried this via asdamin and via the console and get the same result.

Comment by Byron Nevins [ 26/Jun/12 ]

Sorry I overlooked the activity on this issue. I'll try to look into it soon. mtobler – please document what you did/what happened etc. Are you using a Windows Domain?





[GLASSFISH-16311] Improve operating service (OS) integration Created: 04/Apr/11  Updated: 17/Oct/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1
Fix Version/s: future release

Type: Improvement Priority: Critical
Reporter: Tom Mueller Assignee: Byron Nevins
Resolution: Unresolved Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows, Linux, Solaris


Issue Links:
Dependency
depends on GLASSFISH-16525 Add Auto Restarting Capability to Pla... Open
depends on GLASSFISH-10266 provide command to delete services Open
depends on GLASSFISH-11692 create-service improvements Open
depends on GLASSFISH-16522 Add Command for list-services Open
depends on GLASSFISH-16523 Platform Services - Add Support for A... Open
depends on GLASSFISH-16526 Add restart-service command Open
depends on GLASSFISH-16140 server running as a service on Window... Reopened
depends on GLASSFISH-5263 Platform Services and the -Xrs option Resolved
depends on GLASSFISH-12363 The error message of "asadmin create-... Closed
Duplicate
is duplicated by GLASSFISH-17126 -Xrs is missing, so on 100% of all Wi... Closed
Tags: ee7ri_cleanup_deferred

 Description   

This RFE is for improving the operating system (OS) service integration for GlassFish. Here are the requirements:

1. Expose the asadmin delete-service interface as a public interface (rather than being hidden as it is currently).

2. Modify the service implementation so that it acts as a monitor on all operating systems. By monitor, this means that if the service is started, then if the GlassFish process exits, it should be restarted automatically. The Windows service currently doesn't work this way - I'm not sure for Linux and Solaris.

3. Modify the service implementation and/or the various start/stop commands so that they interact correctly. This includes:

3a. If the OS service is started, and the user runs stop-local-instance or stop-domain, then the OS service should be stopped too.

3b. If the server is started using start-domain or start-local-instance, and the OS service is not started, the OS service should not be started. However, if the user then later starts the OS service, the OS service must recognize that the server is already running and monitor the already running server (See req. #2).

4. Any sequence of OS service commands and asadmin start/stop-domain or start/stop-local-instance commands must not cause a failure of the command. For example, currently, if you start the Windows OS service for a domain, and then run stop-domain, and then try to stop the Windows OS service, you get an error message from Windows. Also on Windows, if you run start-domain, and then start the OS service, you get an error message saying that "The process terminated unexpectedly." and the service isn't started.

5. The service should be able to be deleted using either the operating system command for deleting services or the asadmin delete-service command. So the following sequences should work:

asadmin create-service
OS delete service command
asadmin create-service
asadmin delete-service

Currently, the 2nd create-service in this sequence may require a --force option. Ideally, it shouldn't.

6. These requirements apply to the officially supported operating systems for GlassFish for Windows, Linux, and Solaris.



 Comments   
Comment by jclingan [ 04/Apr/11 ]

Good RFE write-up. One additional bullet:

6) Currently we have a feature gap on Linux, where no "watchdog" or "monitor" role is offered using our linux service template. Upstart is available on RHEL 6 and OL 6 (timeline for SuSE Enterprise Linux is TBD). We should investigate if an upstart job definition can be created to fulfill the watchdog role on supported OSs. As a heads-up, it looks like Fedora may move to systemd in the future, so watchdog approach may change in the future.

Comment by Byron Nevins [ 04/Apr/11 ]

Problems with #2
=================== problem #1 ================
Mainly the problem of getting into an infinite loop trying to start an unstable server. Windows allows you to set what to do –
You can set any of the allowed 3 occurrences to handle any of the 4 allowed responses.

1. First Failure
2. Second Failure
3. Subsequent Failures

1. restart the service
2. reboot
3. run some other program
4. ignore it

As you can see just figuring out how to allow users to configure these options and then implementing in 3 main platforms that are totally different is quite a task!

Of course – why is the server crashing? I have V2 running at home now for several years. It never crashes. Do we really want to do all of this complicated and expensive work for something that should be exceedingly rare?

======= problem #2 =========== (This caused many support problems with V2)
User kills the server forcefully (e.g. using "kill" or "taskkill.exe")
A moment later it is running again.
He scratches his head kills it again
A moment later it is running again.
"Hello. Customer support? ...."

Comment by Byron Nevins [ 04/Apr/11 ]

Comment from Bill Shannon about #3B

I agree about 3b. If you start the server without using the OS service
mechanism, you're probably not going to be able to get the service mechanism
to monitor that service later. Mostly this should be as expected. The key
is to get stopping and restarting to integrate properly with the service
mechanism. Possibly also start-instance. I'm not sure we want to make
start-local-instance or start-domain just be front-ends to the service
mechanism (if the server is configured to be handled by the service mechanism).

Comment by Bill Shannon [ 04/Apr/11 ]

Ok, apparently we're going to be using Jira as a discussion forum...

For problem #1, we should just pick some particular combination of
options and allow users to customize it using the OS-specific mechanisms.

For problem #2, this is the normal behavior for any service right?
If the user chooses to have the service mechanism manage his server,
this is what he should expect, and what he would get with any other
server managed by the service mechanism, right? For that matter,
this is what he would get with the old node agent - kill the server
instance and the node agent would restart it because it "crashed",
right?

Comment by Tom Mueller [ 05/Apr/11 ]

The changes to OS service integration should also implement those suggested in issue 11692.

Comment by Tom Mueller [ 05/Apr/11 ]

This issue should resolve issue 16140 also.

Comment by Byron Nevins [ 15/Apr/11 ]

One-Pager Added here:
http://wikis.sun.com/display/GlassFish/3.2PlatformServices

Comment by Byron Nevins [ 15/Apr/11 ]

Pasted the description here. Lines that start with **** are my comments.

1. Expose the asadmin delete-service interface as a public interface (rather than being hidden as it is currently).

        • Yes - Also add start|stop|list

2. Modify the service implementation so that it acts as a monitor on all operating systems. By monitor, this means that if the service is started, then if the GlassFish process exits, it should be restarted automatically. The Windows service currently doesn't work this way - I'm not sure for Linux and Solaris.

        • Yes. I will choose reasonable defaults for each platform – as long as the platform supports it.

3. Modify the service implementation and/or the various start/stop commands so that they interact correctly. This includes:

3a. If the OS service is started, and the user runs stop-local-instance or stop-domain, then the OS service should be stopped too.

        • Misunderstanding of what a Service is. GF-instance and the "service" are the same thing, sort of. Once the service has started - when you stop the server you have stopped the "service". Compare to a SMTP server. When you stop the SMTP server process you have also stopped the SMTP "service".

3b. If the server is started using start-domain or start-local-instance, and the OS service is not started, the OS service should not be started.
****It can't be started. GF won't allow the same server to be started twice.

However, if the user then later starts the OS service, the OS service must recognize that the server is already running and monitor the already running server (See req. #2).

        • Impossible - will not do.

4. Any sequence of OS service commands and asadmin start/stop-domain or start/stop-local-instance commands must not cause a failure of the command. For example, currently, if you start the Windows OS service for a domain, and then run stop-domain, and then try to stop the Windows OS service, you get an error message from Windows. Also on Windows, if you run start-domain, and then start the OS service, you get an error message saying that "The process terminated unexpectedly." and the service isn't started.

        • This is all expected and what we want!

5. The service should be able to be deleted using either the operating system command for deleting services or the asadmin delete-service command. So the following sequences should work:

asadmin create-service
OS delete service command
asadmin create-service
asadmin delete-service

Currently, the 2nd create-service in this sequence may require a --force option. Ideally, it shouldn't.

      • will do

6. These requirements apply to the officially supported operating systems for GlassFish for Windows, Linux, and Solaris.

      • indeed.
Comment by Byron Nevins [ 15/Apr/11 ]

Just thinking out loud here.

1) services actually call

asadmin start-XXX --verbose

2) if the server crashes – asadmin knows about it and is capable of restarting it itself without the platform doing anything at all!

3) if the server is stopped in an orderly way, asadmin knows this also and it can tell the difference from a crash.

==============

Comment by Byron Nevins [ 27/Apr/11 ]

Please see the One Pager:

http://wikis.sun.com/display/GlassFish/3.2PlatformServices

Note that I have added a feature – we will support services on all GlassFish-supported Platforms.

Comment by Byron Nevins [ 02/May/11 ]

THIS IS THE UMBRELLA ISSUE FOR IMPROVED PLATFORM SERVICES for 3.2

Comment by mkarg [ 28/Jul/11 ]

I want to recommend not having two different JVMs involved or using scripts at all on Windows to implement services.

See, on Windows, a real service is implementing an API defined by Microsoft, which lets Windows monitor the service on its own - there is no need for an additional Watchdog, as Windows is a service watchdog (it even comes with configurable rules what to do when the service fails and can restart it etc)! This is the most clean solution and it could be done very easily by just a few lines of JNA code within the GlassFish kernel.

See http://msdn.microsoft.com/en-us/library/ms685141(v=vs.85).aspx

Comment by mkarg [ 28/Jul/11 ]

I want to suggest that asadmin create-service provides a slightly changes configuration:

  • Since Windows 2008 there are special account types for services due to security reasons. I want to suggest that the service is not created to be run as the local SYSTEM account (= with highest possible access rights) but instead the installer should create a local service type account and register on the most essential access rights with that. In a productive system it is not appropriate to run as local SYSTEM account, and the administrator doesn't know what access rights GF will need, so he cannot change it.
  • Windows has a built-in watchdog facility. The configuration should be made up in a way that automatically restarts after first fail, runs some kind of domain repair at the second fail (if there at all is something that GF can repair), and restarts the host at the third fail. The failure counter should be reset after one week. This stuff already is there, so please use it.

Windows is Windows, not UNIX plus a GUI.

Comment by Byron Nevins [ 28/Jul/11 ]

"it could be done very easily by just a few lines of JNA code"

Please to provide these very easy few lines of code.

Comment by Bill Shannon [ 28/Jul/11 ]

Markus, this seems to be important to you, and you seem to know more about it
than most of us. Perhaps you'd be interested in implementing it and contributing
it to the GlassFish community? While we'd all like to see these sorts of
improvements, I doubt that we would do as good a job as you would.

Comment by mkarg [ 29/Jul/11 ]

Bill,

I would be happy to provide an implementation, but I have no strong GlassFish internals background, so I will need help with that. What I can provide is the complete JNA or C++ code for a "good and complete" "real" Windows service, but what I need it someone that will tell me (a) how to build GlassFish from scratch and (b) the few Java lines needed to issue a GF startup / shutdown / status-query. If we can organize this then I will be glad to provide the complete Windows part.

Regards
Markus

Comment by Bill Shannon [ 29/Jul/11 ]

Ideally you would just issue the "asadmin start-domain" command to
start the app server. Is there some reason you need to start it
"in process"? If so, you'll need to duplicate the environment setup
from asadmin.bat and then start a JVM using the same arguments that
asadmin.bat does.

In any event, Byron is the expert on starting GlassFish.

Also, hopefully, you wouldn't need to build GlassFish in order to do
this, but if you did there's build instructions on the wiki.

Comment by Byron Nevins [ 29/Jul/11 ]

I'm not sure what you mean by "the few Java lines needed to issue a GF startup / shutdown / status-query". Perhaps you mean this?

How to start, stop, check status of domain

java -jar "%GF_HOME%\modules\admin-cli.jar" start-domain
java -jar "%GF_HOME%\modules\admin-cli.jar" stop-domain
java -jar "%GF_HOME%\modules\admin-cli.jar" list-domains

How to start, stop, check status of instance

java -jar "%GF_HOME%\modules\admin-cli.jar" start-local-instance instance1
java -jar "%GF_HOME%\modules\admin-cli.jar" stop-local-instance instance1
java -jar "%GF_HOME%\modules\admin-cli.jar" list-instances --long

------------------
If you mean the source lines that run from the above calls – they are definitely not "few". There are thousands of lines required. They are located in admin/launcher, core/kernel, core/bootstrap, cluster/cli, cluster/admin and common/common-util (off the top of my head)

Comment by mkarg [ 30/Jul/11 ]

My idea is basing on in-process because it is the natural way on Windows to implement services, and it simplifies the complexity by far, as there is no more watchdog asadmin needed. GlassFish will just feel and behave as a native service, so no more scripts are involved. Windows admins don't like scripts, as Windows has a completely different architecture compared to UNIX. UNIX does everything in scripts, Windows does virtually nothin in scripts. So the target is, to get rid of scripts.

Thank you Byron for the hint. Actually I looked for the single entry points in Java source code that make the following happen (in pseudo code):

  • GlassFish.start!
  • GlassFish.stop!
  • (opt.) GlassFish.pause!
  • (opt.) GlassFish.resume!
  • GlassFish.state?

If that is not existing, I will inspect what the scripts do and repeat that in pure Java (hence my question about building from scratch; got that meanwhile using svn and mvn BTW). My target is to provide java source that is using / implementing the native Windows API that directory executes this commands in-process.

Comment by Tom Mueller [ 17/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issues was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-11115] dtrace shouldn't list hidden probes Created: 20/Nov/09  Updated: 18/Oct/12

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: V3
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: sankarpn Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: PC


Issuezilla Id: 11,115
Status Whiteboard:

3.1-exclude v3_exclude

Tags: ee7ri_cleanup_deferred

 Description   

Filing this as per this email conversation.

Sankar,

Can you please file a bug to track this.

thanks
Prashanth

Byron Nevins wrote:
> Agreed – yes a bug. Significant amount of work to fix it – i.e. the code
that uses ASM to generate classes at runtime for DTrace have to be changed to
not produce those hidden methods. We should report it for 3.1
>
> I already have a fix for the case where the entire ProbeProvider is all hidden
(totally ignore the entire class) – but Jerome did not approve it for post HCF
checkin.
>
>
> Prashanth Abbagani wrote:
>
>> Thats a bug, showing something to the user which doesn't work.
>>
>>
>> Byron Nevins wrote:
>>
>>> Ironically, dtrace will list all hidden probes. The will never fire into
DTrace at runtime though...
>>>
>>>
>>> Marina Vatkina wrote:
>>>
>>>> Prashanth Abbagani wrote:
>>>>
>>>>> Marina Vatkina wrote:
>>>>>
>>>>>> Prashanth Abbagani wrote:
>>>>>>
>>>>>>> I guess almost all the time, the toString() is useless for the user when
we expose the complex types. Imagine what a Throwable.toString() would mean to
them when they are sitting in front of a console. For this reason, its better to
restrict the probes to expose only the types that make sense (Primitive and
Strings).
>>>>>>>
>>>>>>> This is not the same for javascript client though, as the user can
programmatically make sense of that object.
>>>>>>
>>>>>>
>>>>>>
>>>>>> How would a javascript user know about the probes if list-probes doesn't
list them?
>>>>>
>>>>>
>>>>> The restriction of probes to be kept hidden if they are not of type
primitive, is due to the DTrace user. I am making it clear that it has no
restrictions with other clients.
>>>>
>>>>
>>>> So it's a bug that list-probes doesn't list probes mark as hidden. Is it?
>>>>
>>>>>
>>>>>>
>>>>>> thanks,
>>>>>> -marina
>>>>>>
>>>>>>>
>>>>>>> Mahesh Kannan wrote:
>>>>>>>
>>>>>>>> Interesting, then why do we have this hidden attribute?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> --Mahesh
>>>>>>>>
>>>>>>>>
>>>>>>>> Byron Nevins wrote:
>>>>>>>>
>>>>>>>>> I keep hearing that and it is /not true/. The DTrace code in
Flashlight handles any and all types.
>>>>>>>>> Of course if the class does not bother implementing toString() – a
useless String will be produced...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Marina Vatkina wrote:
>>>>>>>>>
>>>>>>>>>> Were they working? Mahesh made them hidden because dtrace can't
handle their param types (Method and Throwable).
>>>>>>>>>>
>>>>>>>>>> It also doesn't seem right not to list probes if dtrace can't handle
them.
>>>>>>>>>>
>>>>>>>>>> thanks,
>>>>>>>>>> -marina
>>>>>>>>>>
>>>>>>>>>> Sankar Neelakandan wrote:
>>>>>>>>>>
>>>>>>>>>>> What happened to these probes ?. I no loner see it in list-probes
or dtrace -l.
>>>>>>>>>>> However when I register it through scripting client the probes are
fired. But not through dtrace script.
>>>>>>>>>>> Is there any changes went into these probes ?.
>>>>>>>>>>>
>>>>>>>>>>> glassfish:ejb:bean:methodEndEvent
>>>>>>>>>>> glassfish:ejb:bean:methodStartEvent
>>>>>>>>>>>
>>>>>>>>>>>
count=84,probe=glassfish:ejb:bean:methodStartEvent,appName=82487359949242370,modName=ejbsfapp1,ejbName=ejbsfapp1ejbmod1_jar,method=SFApp1EJB1
>>>>>>>>>>>
>>>>>>>>>>>
count=212,probe=glassfish:ejb:bean:methodEndEvent,appName=ejbsfapp1,modName=ejbsfapp1ejbmod1_jar,ejbName=SFApp1EJB1,exception=null,method=public
abstract
examples.monitoring.ejbapps.stateful.ejbsfapp1.ejbsfapp1ejbmod1.SFApp1EJB1
examples.monitoring.ejbapps.stateful.ejbsfapp1.
>>>>>>>>>>> ejbsfapp1ejbmod1.SFApp1EJB1Home.create(java.lang.String) throws
javax.ejb.CreateException,java.rmi.RemoteException
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> –
>>>>>>>>> Byron Nevins - Sun Microsystems, Inc.
>>>>>>>>> Home: 650-359-1290
>>>>>>>>> Cell: 650-784-4123
>>>>>>>>> Sierra: 209-295-2188
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>



 Comments   
Comment by dochez [ 20/Nov/09 ]

btrace is not a supported feature of v3.

Comment by kumara [ 07/Dec/09 ]

Setting target release for unresolved issues submitted on v3 release to the next release. Not changing
issues submitted on v2.x release because they might not apply to v3.next release.

Comment by Nazrul [ 14/May/10 ]

-> Byron

Comment by Tom Mueller [ 18/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issue was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-15167] Unable to step through Flashlight Framework (probes infrastructure) code with Netbeans debugger Created: 14/Dec/10  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3_1-exclude

 Description   

Set a break point in Flashlight Framework code and try to step through. It will crash.
It would be nice to be able to step through for debugging issues like
http://monaco.sfbay.sun.com/detail.jsf?cr=7004360



 Comments   
Comment by Nazrul [ 20/Dec/10 ]

Excluding from 3.1 count

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-10393] Add MacOS X service support Created: 19/Oct/09  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: V3
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: ralphrmartin Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Macintosh


Issuezilla Id: 10,393

 Description   

There needs to be some way to install glassfish so that

(a) it runs automatically at boot time, via Apple's /Library/LaunchDaemon mechanism

(b) it installs and runs as some user like _www rather than as a logged in user.



 Comments   
Comment by Snjezana Sevo-Zenzerovic [ 19/Oct/09 ]

This really translates into adding MacOS support for 'asadmin create-service'
command. Adjusting the summary and moving to "admin" subcategory.

Comment by km [ 01/Nov/09 ]

bn

Comment by Byron Nevins [ 08/Mar/10 ]

I need access to a Macintosh

Comment by Byron Nevins [ 08/Mar/10 ]

I need access to a Macintosh

Comment by Byron Nevins [ 11/Mar/10 ]

No response – assuming not approved for 3.0.1

Comment by Byron Nevins [ 28/Sep/10 ]

3.2

Comment by ralphrmartin [ 28/Sep/10 ]

This is just receding into the distance. Why?

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-10266] provide command to delete services Created: 14/Oct/09  Updated: 17/Oct/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: V3
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: sankarpn Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Solaris
Platform: PC


Issue Links:
Dependency
blocks GLASSFISH-16311 Improve operating service (OS) integr... Open
Issuezilla Id: 10,266
Tags: ee7ri_cleanup_deferred

 Description   

currently there is command to create the service but there is no commands to
delete/list the service entries without the user interacting with the OS. Would
be nice to have this feature.



 Comments   
Comment by Bill Shannon [ 14/Oct/09 ]

I think I made the same comment some time ago...

Comment by Byron Nevins [ 14/Oct/09 ]

Let's see if a day frees up before nov 9 to do this. It is probably worthwhile.
I forget the intricate OS commands almost instantly after running create-service.

It would add much value...

bumping to P3/defect for now.

Comment by Byron Nevins [ 26/Oct/09 ]

Changed to RFE

Probably won't be time to implement and document for 3.0

Comment by Byron Nevins [ 25/Jan/10 ]

Increased priority etc.

Comment by kumara [ 04/Feb/10 ]

Not required for v3.0.1 and therefore being retargeted to v3.1

Comment by Byron Nevins [ 06/Jul/10 ]

ms4

Comment by Byron Nevins [ 18/Aug/10 ]

No time for ms4.
I'm setting to 3.1 in case time is available post ms5

Comment by Nazrul [ 28/Oct/10 ]

Will consider in 3.2

Comment by Byron Nevins [ 01/Apr/11 ]

feature not bug..

Comment by Byron Nevins [ 15/Apr/11 ]

also stop and start

Comment by Byron Nevins [ 02/May/11 ]

Changed this issue to be just for delete-service

Comment by Tom Mueller [ 17/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issues was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-11692] create-service improvements Created: 17/Mar/10  Updated: 17/Oct/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Solaris
Platform: All


Attachments: Text File solaris_issues.txt    
Issue Links:
Dependency
blocks GLASSFISH-16311 Improve operating service (OS) integr... Open
Issuezilla Id: 11,692
Tags: ee7ri_cleanup_deferred

 Description   

Excellent comments from Jens Elkner.
See attached file for his entire email

Excerpts about SMF in particular:

================================

(5.0) Create the SMF service as root or privileged user (RBAC SW admin)
using the command line (i.e. no GUI!!!)
Command:
+ $INSTALL_HOME/bin/asadmin create-service \
--domaindir $DATA_DIR $DOMAIN
Result:
a) if /var/svc/manifest/application/GlassFish does not yet
exist, create it with ownership root:sys and 0755
permissions.
b) MANIFEST=/var/svc/manifest/application/GlassFish/$DOMAIN.xml
with ownership root:sys and 0644 permissions. If that file
already exists, bail out with an appropriate error message
(e.g. $MANIFEST already exists. Please remove that service
first or choose another name ...)
c) $MANIFEST's start and stop method context should have
appropriate values, which reflect this instance. I.e.

  • working_directory = $DATA_DIR/$DOMAIN
  • method_credential user="$AS_USER"
    NOTE: since the instance must be able to write to its log
    directory, $AS_USER can be deduced from the ownership of
    $DATA_DIR/$DOMAIN/logs, i.e. there is not really a need
    for an --asuser switch wrt. create-service cmd (but might
    be beneficial for other use cases).
  • set privileges unconditionally to:
    "basic,!proc_session,!proc_info,!file_link_any"
  • if $AS_USER != "root" && $privatePortsInUse add
    'net_privaddr' to the privileges unconditionally
    NOTE: even if $AS_USER == "root" GF should never run with
    all available OS/role/user privileges. For the rare
    cases, where add. privileges are really required,
    they should be explicitly added by editing $MANIFEST
    or passing them via a TBD switch (e.g.
    --addprivs=proc_info[:...])
    d) add an appropriate property group to the $MANIFEST's instance.
    E.g.
    <property_group name='general' type='framework'>
    <propval name='action_authorization' type='astring'
    value='solaris.smf.manage.glassfish' />
    <propval name='value_authorization' type='astring'
    value='solaris.smf.manage.glassfish' />
    </property_group>
    For more fine grained permissions,
    solaris.smf.manage.glassfish.$DOMAIN might be also an option
    e) Optional: document the new auths if not already done. E.g.:
    echo 'solaris.smf.manage.glassfish.:::GF Management' \
    >>/etc/security/auth_attr,
    emit a message, that the new auths (which) have been created
    f) import the $MANIFEST
    g) emit svcadm|svccfg hints

(5.1) As an alternative to (5.0) - Create an appropriate Manifest ready
for SMF import as unprivileged user (e.g. $AS_USER) using the
command line (i.e. no GUI!!!)
Command:
$INSTALL_HOME/bin/asadmin create-service \
[--out $file] \
--domaindir $DATA_DIR $DOMAIN
Result:
a) if --out is given, set MANIFEST=$file
otherwise use MANIFEST=$DATA_DIR/$DOMAIN/manifest.xml
b) same as (5.0) c)
c) same as (5.0) d)
d) emit a hint, where one should put the manifest and how to
import it. E.g.:
'Please ask your system administrator to import the SMF
manifest "$MANIFEST", so that this instance gets
automatically stopped/started on system shutdown/boot
respectively using the following commands:

cp $MANIFEST /var/svc/manifest/application/GlassFish/$DOMAIN.xml
svccfg import /var/svc/manifest/application/GlassFish/$DOMAIN.xml
svcadm enable glassfish/$DOMAIN

Also you may ask your Admin to allow you to manage glassfish
instances by adding solaris.smf.manage.glassfish.* to your
auths. E.g.:
usermod -A 'solaris.smf.manage.glassfish.*,'`auths $USER` $USER
'
e) same as (5.0) g)

========================
c) Assuming the unprivileged user got its domain created (e.g. by
using the checkports option, asking the admin, etc.)
the next illness gets uncovered: E.g:

/opt/glassfish/v3/bin/asadmin create-service \
--domaindir /data/sites/glassfish/v3 jforum

The user [webservd] does not have permission to create the service
manifest related files and directories at
[/var/svc/manifest/application/GlassFish/]. This structure is required
per SMF guidelines. Either become super-user to do this operation or
contact the System Administrator to explicitly get the relevant
permissions and try again.
Usage: asadmin [asadmin-utility-options] create-service [--name <name>]
[--serviceproperties <serviceproperties>]
[--dry-run[=<dry-run(default:false)>]] [--domaindir <domaindir>]
[?|-help[=<help(default:false)>]] [domain_name]
Command create-service failed.

First hmm: Why is this message spoiled with the usage info?
2nd hmmm: A more or less relaxed admin might say - oh know,
don't bother me all the time with your crap:
mkdir -p /var/svc/manifest/application/GlassFish/
chown student:sgid /var/svc/manifest/application/GlassFish
and tell him 'send me an email, when you changed something'.

Student tries it again:

The user [student] does not seem to have adequate authorizations
[solaris.smf.*] on this System to create and configure an SMF service.
The authorizations available are
[solaris.device.cdrw,solaris.profmgr.read,solaris.jobs.users,solaris.mail.mailq,solaris.admin.usermgr.read,solaris.admin.logsvc.read,solaris.admin.fsmgr.read,solaris.admin.serialmgr.read,solaris.admin.diskmgr.read,solaris.admin.procmgr.user,solaris.compsys.read,solaris.admin.printer.read,solaris.admin.prodreg.read,solaris.admin.dcmgr.read,solaris.snmp.read,solaris.project.read,solaris.admin.patchmgr.read,solaris.network.hosts.read,solaris.admin.volmgr.read
].
See smf_security(5), rbac(5).

Usage: asadmin [asadmin-utility-options] create-service [--name <name>]
[--serviceproperties <serviceproperties>]
[--dry-run[=<dry-run(default:false)>]] [--domaindir <domaindir>]
[?|-help[=<help(default:false)>]] [domain_name]
Command create-service failed.

First hmmm: Why gets the message spoiled with the usage info?
2nd hmmm: Oh my goodness! I need to bother the admin again.
3rd hmm: the admin starts asking itself, what kind of strange
software is it, which requires solaris.smf.* just to be able to
create a manifest ...

So the SW designer made another fault: He tries to solve
two different problems at once (create the manifest, import it).

IMHO the correct and less annoying way is:
1) create the manifest as described in (5.0) c) and d)
2) try to copy it to /var/svc/manifest/application/GlassFish/$DOMAIN.xml
if that fails
a) store it as $DATA_DIR/$DOMAIN.xml
b) emit a warning 'Unable to save manifest as ....'
and a message as suggested in (5.1) d) and exit
3) try to import the manifest
if that fails, emit a warning like
'Unable to import manifest $MANIFEST (insufficient auths).' and
add a message similar to (5.1) d) but without the copy instruction
and exit

The hint wrt. /var/svc/manifest/application/GlassFish/ permission
is IMHO completely misleading and dangerous as well - so vaporize
it completely!

Furthermore the error message wrt. to solaris.smf.* auths required
is also misleading. The unexperienced user/admin @home interpretes
this actually as an instruction to assign those auths to the given
user, which is plain wrong (analog to the net_privaddr problem).



 Comments   
Comment by Byron Nevins [ 17/Mar/10 ]

Created an attachment (id=4272)
Email Comments

Comment by Byron Nevins [ 06/Jul/10 ]

ms4

Comment by Byron Nevins [ 17/Aug/10 ]

Not enough time to do this for MS4.

Set to MS5 for now...

Comment by Tom Mueller [ 17/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issues was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-11308] Need addition support for XML Probe Providers Created: 13/Dec/09  Updated: 18/Oct/12

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: abbagani Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Macintosh


Issuezilla Id: 11,308
Status Whiteboard:

3.1-exclude

Tags: ee7ri_cleanup_deferred

 Description   

Need additional monitoring infrsastructure support for ProbeProviders written in
XML. Should be able to insert the Probe in a method, towards the end of the
method, at any line number or when the method throws an exception. Currently it
only inserts at the beginning of the method.



 Comments   
Comment by Nazrul [ 14/May/10 ]

-> Byron

Comment by Tom Mueller [ 18/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issue was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-10076] start-domain: support reactive timeout Created: 07/Oct/09  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: V3
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 10,076

 Description   

We currently arbitrarily set a "one-size fits all" timeout in start-domain.

I suggest we cool-ify it like so:

1) Start with the usual timeout – 600 seconds or whatever.
2) For each of the very first 10 or 20 starts just note the actual start time in
a special file in the config directory or log directory.

    • 10-20 allows them some time to make their system more complicated.
    • Or we wait to see a pattern where the start times start to be +- 20% of each
      other
      3) Double that amount of time and make that the timeout
      4) If it times out we say something smart like:
      "Based on your domain's history it takes an average of 17 seconds to start. The
      domain failed to start in 34 seconds. Check the server log for problems.
    • we now reset to (1) and start collecting history again...

As a bonus this would be very useful for QE/Performance. We produce a file
automatically with startup times.



 Comments   
Comment by km [ 01/Nov/09 ]

bn

Comment by Paul Davies [ 18/Feb/11 ]

Consider also adding an option like --timeout to start-domain to give users some control over this behavior.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-13130] Allow attaching btrace scripts to server Created: 26/Aug/10  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: Sanjeeb Sahoo Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issue Links:
Dependency
blocks GLASSFISH-13720 parallel issue for btrace Open
Issuezilla Id: 13,130

 Description   

Currently although GF uses btrace for its monitoring implementation, it does not
expose it to end users. In fact, the btrace agent is started in noServer mode to
avoid security risks, so one can't use btrace client to connect to server. See
[1]. At the very least, we should start the agent in server mode when debugging
is ON. We can also add a new module (not part of default distribution), which
exposes btrace to end users in one or more of the following ways:

a) have a configurable watched directory for btrace scripts,
b) have an asadmin command to attach a btrace script.

[1]
http://weblogs.java.net/blog/jjviana/archive/2010/07/14/using-btrace-glassfish-v3



 Comments   
Comment by Sanjeeb Sahoo [ 23/Mar/11 ]

I have no idea why this bug was still owned by me. Reassigning to module owner for further action.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-12088] Asadmin should handle asenv.conf AS_JAVA value which is relative path Created: 01/Jun/10  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: command_line_interface
Affects Version/s: 3.1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Snjezana Sevo-Zenzerovic Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 12,088

 Description   

Spillover from issue 12083. Asadmin scripts (and possibly some of asadmin JDK
detection code, too) should be enhanced to correctly handle relative value of
AS_JAVA asenv config file variable. This will feature will primarily be used for
Java EE SDK distributions with bundled JDK.



 Comments   
Comment by Bill Shannon [ 01/Jun/10 ]

Assign to Byron. I think it's the launcher that would handle this.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-12046] Add option to stop-domain that would stop clusters/instance Created: 25/May/10  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Tom Mueller Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 12,046

 Description   

This is an enhancement request for adding an option to stop-domain that would
automatically run stop-cluster for every cluster (which would stop the instances
in any cluster) and run stop-instance for every stand-alone instance that is not
in a cluster. With this option, a user would be able to bring down every
instance that is associated with a domain using a single command.

Several possible options for the option might be: --stop-instances or
--include-all-instances



 Comments   
Comment by Byron Nevins [ 28/Sep/10 ]

3.2

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-15590] Regular Expression Checking not working for calculated values Created: 17/Jan/11  Updated: 18/Feb/13

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: future release

Type: Bug Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3_1_1-exclude, 3_1_1-scrubbed, 3_1_x-exclude

 Description   

D:\gf\v3\core\kernel>asadmin get -m server.applications.x.y\.z.server.jsp.servicetime-coun*
No monitoring data to report.

D:\gf\v3\core\kernel>asadmin get -m server.applications.x.y\.z.server.jsp.servicetime-count
server.applications.x\.y\.z.server.jsp.servicetime-count = 0

=======
The first command ought to work.



 Comments   
Comment by Byron Nevins [ 17/Jan/11 ]

Another example:
========================
D:\gf\v3\core\kernel>asadmin get -m server.applications.x.y.z.server.jsp.servicetime*

server.applications.x\.y\.z.server.jsp.dotted-name =
// lots of output removed //

============
D:\gf\v3\core\kernel>asadmin get m server.applications.x.y.z.server.jsp.servicetime*
No monitoring data to report.

Comment by Byron Nevins [ 22/Mar/11 ]

As you can see – the internal code is NOT looking at things as final strings and thus misses stuff.

Another Example

D:\>asadmin get -m server.applications.x\.y.server.jsp.servicetime*
server.applications.x\.y.server.jsp.dotted-name = server.applications.x\.y.server.jsp
server.applications.x\.y.server.jsp.servicetime-count = 0
server.applications.x\.y.server.jsp.servicetime-description = Aggregate response time
server.applications.x\.y.server.jsp.servicetime-lastsampletime = 1300811411960
server.applications.x\.y.server.jsp.servicetime-name = ServiceTime
server.applications.x\.y.server.jsp.servicetime-starttime = 1300811252183
server.applications.x\.y.server.jsp.servicetime-unit = millisecond

D:\>asadmin get m server.applications.x\.y.server.jsp.servicetime*
No monitoring data to report.

Comment by Byron Nevins [ 14/Apr/11 ]
  • Why fix this issue in 3.1.1?
    Just piping output to grep with a regular expression works. Meanwhile giving the same regular expression to the monitoring framework gets no matches at all – this reflects badly on the product.
  • Which is the targeted build of 3.1.1 for this fix?
    I have no list of builds to pick from ?!?
  • Do regression tests exist for this issue?
    Yes. There are many unit tests, dev tests and QE tests
  • Which tests should QA (re)run to verify the fix did not destabilize GlassFish?
    Homer should re-run the QE 3.1 monitoring tests
Comment by scatari [ 14/Apr/11 ]

Approved.

Comment by Byron Nevins [ 24/May/11 ]

Excluding from 3.1.1

I spent an hour looking at this. The code has "high resistance to change". It is too risky and time-consuming for 3.1.1

Also there is a very very easy work-around which is to move the star upstream.

E.g. this does not work because the "-count" is a calculated thing. There is no "real" monitoring data with that name ending in a hyphen

server.applications.HelloWeb.server.sessionstotal-*

But this works fine:

server.applications.HelloWeb.server.sessionstotal*





[GLASSFISH-15537] restart-instance takes a long time on Solaris 11 (sun.security.pkcs11.SunPKCS11) Created: 11/Jan/11  Updated: 18/Feb/13  Due: 18/Apr/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1
Fix Version/s: future release

Type: Bug Priority: Major
Reporter: Joe Di Pol Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Oracle Solaris 11 Express snv_151a X86
java version "1.6.0_23"
Private build based on r44408


Attachments: File rpc.c    
Tags: 3_1-exclude, 3_1-next_release-note-added, 3_1-next_release-notes, 3_1_1-scrubbed, 3_1_x-exclude

 Description   

restart-instance often takes minutes to execute on Solaris 11. To reproduce:

  1. Start the DAS. On the DAS machine:
    asadmin create-local-instance i1
    asadmin start-local-instance i1
    asadmin stop-local-instance i1
    asadmin start-local-instance i1
  1. So far, so good. This all works great. I can stop/start
  2. many times and I don't see the problem. Then do:

asadmin restart-instance i1

Sometimes this returns promptly. But if I run restart repeatedly
often it takes a while to return (one time 9 minutes).
The instance does start eventually. If I do a jstack of
the instance process while it is "hung" I see consistently this
thread which appears to be blocked in a pkcs11 call (see stack
trace below).

I have not been able to reproduce this on Linux.

"FelixStartLevel" daemon prio=3 tid=0x089a5400 nid=0xe runnable [0xcd17e000]
java.lang.Thread.State: RUNNABLE
at sun.security.pkcs11.wrapper.PKCS11.C_Initialize(Native Method)
at sun.security.pkcs11.wrapper.PKCS11.getInstance(PKCS11.java:160)

  • locked <0xcebc0c58> (a java.lang.Class for sun.security.pkcs11.wrapper.PKCS11)
    at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:281)
    at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:86)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at sun.security.jca.ProviderConfig$3.run(ProviderConfig.java:243)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:225)
    at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:205)
  • locked <0xf1e8f838> (a sun.misc.Launcher$AppClassLoader)
    at sun.security.jca.ProviderList.getProvider(ProviderList.java:215)
    at sun.security.jca.ProviderList.getService(ProviderList.java:313)
    at sun.security.jca.GetInstance.getInstance(GetInstance.java:140)
    at java.security.Security.getImpl(Security.java:659)
    at java.security.MessageDigest.getInstance(MessageDigest.java:129)
    at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1759)
    at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:52)
    at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:205)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:202)
    at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:558)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1583)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1496)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
    at java.util.HashMap.readObject(HashMap.java:1030)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
    at org.jvnet.hk2.osgiadapter.OSGiModulesRegistryImpl.loadCachedData(OSGiModulesRegistryImpl.java:195)
    at org.jvnet.hk2.osgiadapter.OSGiModulesRegistryImpl.<init>(OSGiModulesRegistryImpl.java:91)
    at org.jvnet.hk2.osgiadapter.OSGiFactoryImpl.createModulesRegistry(OSGiFactoryImpl.java:70)
    at org.jvnet.hk2.osgiadapter.OSGiFactoryImpl.createModulesRegistry(OSGiFactoryImpl.java:52)
    at org.jvnet.hk2.osgiadapter.HK2Main.createModulesRegistry(HK2Main.java:185)
    at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:142)
    at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:629)
    at org.apache.felix.framework.Felix.activateBundle(Felix.java:1827)
    at org.apache.felix.framework.Felix.startBundle(Felix.java:1744)
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:922)
    at org.glassfish.kernel.GlassFishActivator.startBundle(GlassFishActivator.java:239)
    at org.glassfish.kernel.GlassFishActivator.startBundles(GlassFishActivator.java:194)
    at org.glassfish.kernel.GlassFishActivator.start(GlassFishActivator.java:80)
    at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:629)
    at org.apache.felix.framework.Felix.activateBundle(Felix.java:1827)
    at org.apache.felix.framework.Felix.startBundle(Felix.java:1744)
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:922)
    at org.jvnet.hk2.osgimain.Main.start(Main.java:154)
    at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:629)
    at org.apache.felix.framework.Felix.activateBundle(Felix.java:1827)
    at org.apache.felix.framework.Felix.startBundle(Felix.java:1744)
    at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1148)
    at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:264)
    at java.lang.Thread.run(Thread.java:619)

The instance log file has nothing of interest. The message before the "hang" is:

Jan 11, 2011 5:21:30 PM com.sun.enterprise.admin.launcher.GFLauncherLogger info
INFO: Successfully launched in 39 msec.

and the first message after the "hang" is:

Jan 11, 2011 5:25:02 PM null
INFO: Running GlassFish Version: GlassFish Server Open Source Edition 3.1-SNAPSHOT (build dipol-private)
[#|2011-01-11T17:25:02.483-0800|INFO|glassfish3.1|null|_ThreadID=12;_ThreadName=Thread-1;|Grizzly Framework 1.9.28 started in: 89ms - bound to [0.0.0.0:28080]|#]



 Comments   
Comment by Chris Kasso [ 12/Jan/11 ]

I'm able to reproduce this as well on S11. It occurred once in seven restarts (it took six minutes to restart):

ouch: time ./asadmin restart-instance i1
i1 was restarted.
Command restart-instance executed successfully.

real 0m19.39s
user 0m16.59s
sys 0m0.83s
ouch: time ./asadmin restart-instance i1
i1 was restarted.
Command restart-instance executed successfully.

real 0m32.63s
user 0m19.04s
sys 0m0.83s
ouch: time ./asadmin restart-instance i1
i1 was restarted.
Command restart-instance executed successfully.

real 6m41.79s
user 0m21.63s
sys 0m1.22s
ouch: cd v3
ouch: time ./asadmin restart-instance i1
i1 was restarted.
Command restart-instance executed successfully.

real 0m32.51s
user 0m19.70s
sys 0m1.26s
ouch: time ./asadmin restart-instance i1
i1 was restarted.
Command restart-instance executed successfully.

real 0m31.14s
user 0m18.31s
sys 0m0.99s
ouch: time ./asadmin restart-instance i1
i1 was restarted.
Command restart-instance executed successfully.

real 0m22.61s
user 0m20.44s
sys 0m0.85s
ouch: time ./asadmin restart-instance i1
i1 was restarted.
Command restart-instance executed successfully.

real 0m56.26s
user 0m19.77s
sys 0m0.81s

Comment by Byron Nevins [ 12/Jan/11 ]

Solaris 11 machine I don't have.

Also – the hung thread is in Felix. Hint.

Joe/Chris - Please try again using no OSGi and then Equinox.

Comment by Joe Di Pol [ 12/Jan/11 ]

I set

GlassFish_Platform=Static

and could still reproduce the problem. The thread trace is now different. It looks like the first call that causes the PKCS11 security provider to be initialized triggers the problem. The weird thing is that this only seems to happen on re-start. I haven't seen this when starting an instance.

The stack trace for when Felix is out of the loop:

"main" prio=3 tid=0x08072c00 nid=0x2 runnable [0xfe2fd000]
java.lang.Thread.State: RUNNABLE
at sun.security.pkcs11.wrapper.PKCS11.C_Initialize(Native Method)
at sun.security.pkcs11.wrapper.PKCS11.getInstance(PKCS11.java:160)

  • locked <0xcef09e80> (a java.lang.Class for sun.security.pkcs11.wrapper.PKCS11)
    at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:281)
    at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:86)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at sun.security.jca.ProviderConfig$3.run(ProviderConfig.java:243)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:225)
    at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:205)
  • locked <0xda6007f8> (a sun.misc.Launcher$AppClassLoader)
    at sun.security.jca.ProviderList.getProvider(ProviderList.java:215)
    at sun.security.jca.ProviderList$3.get(ProviderList.java:130)
    at sun.security.jca.ProviderList$3.get(ProviderList.java:125)
    at java.util.AbstractList$Itr.next(AbstractList.java:345)
    at java.security.SecureRandom.getPrngAlgorithm(SecureRandom.java:522)
    at java.security.SecureRandom.getDefaultPRNG(SecureRandom.java:165)
    at java.security.SecureRandom.<init>(SecureRandom.java:133)
    at

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
com.sun.enterprise.v3.admin.LocalPasswordImpl.postConstruct(LocalPasswordImpl.java:87)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:128)
at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:88)
at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:79)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:64)

  • locked <0xf6fe29b8> (a com.sun.hk2.component.SingletonInhabitant)
    at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:136)
    at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:73)
    at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:219)
    at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:135)
  • locked <0xfa648f10> (a com.sun.enterprise.v3.server.AppServerStartup)
    at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79)
  • locked <0xfa648ef0> (a com.sun.enterprise.glassfish.bootstrap.StaticGlassFishRuntime$1)
    at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97)
    at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55)
Comment by Byron Nevins [ 12/Jan/11 ]

From the internet:

No it hasn't anything to do with the classpath I've used...

I've investigated the problem a little bit further by profiling my
simple jdbc test class and found out that most of the time is spent in
this class:

sun.security.pkcs11.wrapper.PKCS11.C_Initialize

98% of the time is spent in that function with 3694 calls (well it
will last forever if I don't kill the process)!

So I guess it's a problem with Sun/Solaris security mixed with nfs issue...

Christian

On Feb 16, 2008 4:09 PM, Dave Cramer <pg(at)fastcrypt(dot)com> wrote:
>
> On 16-Feb-08, at 2:45 PM, Christian Bourque wrote:
>
> > Hi Tom,
> >
> > I think you were right, there is something in the home directory that
> > is conflicting!
> >
> > I don't understand why jdbc would use something in the user's home
> > directory though...
> >
> > Anyway the workaround I found was to not map the share directly under
> > the user's home account and now it works!
> >
> My guess is your classpath ?
>
> Dave
> > Thanks for your help!
> >
> > Christian
> >
> > On Feb 16, 2008 1:19 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> "Christian Bourque" <christian(dot)bourque(at)gmail(dot)com> writes:
> >>> The pg server is running on server B (openSUSE) and I'm connecting
> >>> to
> >>> it from server C (Solaris 10) using a user account with an nfs
> >>> mounted
> >>> directory from server A (openSUSE).
> >>
> >>> If I unmount the nfs share it works! But with the share it doesn't,
> >>> the connection seems to reach the server but it gets stuck there, no
> >>> error and it doesn't timeout either...
> >>
> >> It's way too hard to believe that NFS per se is interfering.
> >>
> >> What I could believe is that there is some configuration-type file in
> >> your home directory that is causing a problem, and after the unmount
> >> it's not visible so no problem. There are obvious possibilities for
> >> this such as ~/.psqlrc if you're using psql/libpq, but I dunno enough
> >> about the JDBC environment to guess whether it has equivalents.
> >>
> >> If all else fails, you could try strace'ing the application (or
> >> whatever
> >> Solaris' equivalent to strace is) in both cases and comparing
> >> results.
> >> That would at least make it clearer where it's hanging up ...
> >>
> >> regards, tom lane
> >>
> >

Comment by Byron Nevins [ 12/Jan/11 ]

My Analysis:

Facts:

  • PKCS native code has an issue in Solaris 11
  • The problem never happens Solaris < 11 or any other OS
  • The problem never happens with start-domain – only with restart
  • It is intermittent
  • It has nothing to do with LocalPasswordImpl – it hangs with the first of many calls from GF
    to PKCS code. LocalPasswordImpl happens to be the first when running w/o OSGi

Conclusions:

  • It's a race condition inside the native code. The restarting server is coming up too fast for the native code to completely finish. Sometimes.

Fix:
restart-server should wait a bit longer before starting – at least in this one platform.

Comment by Byron Nevins [ 13/Jan/11 ]

======== Here is how restart works in this bug's case

1) Inside the server a new JVM process is spawned to run start-domain
2) The server continues dying
3) Meantime the new JVM is running start-domain from AsadminMain
4) The new JVM waits for its parent process to die
5) start-domain then proceeds as normal

step (4) is difficult. Impossible without using native calls. Here is how it does it now:

(1) It waits for a read call on System.in to return -1.
(2) It positively checks all the ports the server was using

When all the ports are free – we define the server as dead.
==================================================================

THe next step would be to absolutely positively verify that the process itself is gone. Luckily we have ProcessUtils available!

Comment by Byron Nevins [ 13/Jan/11 ]

PROBLEM:

In Solaris 11 restart-domain intermittently hangs. This is caused by Security SPI native code from sun that is included in the JDK.

It has only been observed on Solaris 11 systems at a ~~ 10-15% frequency

How bad is its impact? (Severity)
When it hits – the server hangs and now you can never restart it again. I assume it requires a forcedful kill.

How often does it happen?
10-15% on Sol 11

Will many users see this problem? (Frequency)
The propertion of users using any kind of Solaris is small, I assume. The proportion of them that are using Sol 11 will be small but growing over time.

How much effort is required to fix it? (Cost)
2-3 days. A lot of time-consuming testing, getting remote machines configured, etc. It's intermittent so tests have to be run over and over and over...

What is the risk of fixing it and how will the risk be mitigated? (Risk)
The risk is that restart-server stops working on other platforms
Mitigated by being very careful & conservative. I will probably end up with special
code that only runs on Sol 11 machines.

Comment by Joe Di Pol [ 13/Jan/11 ]

Here is some output from pstack on a "hung" instance. I also ran pfiles, but that didn't show much. I'm not sure what is going on, but some observations:

  • The trace has sendTCSDPacket() on it
  • tcsd(8) is a daemon that acts as the portal to the TPM (Trusted Platform Module) device driver.
  • I think the TPM device driver is how hardware crypto devices are accessed.
  • tcsd(8) is not running on my system because there is no /dev/tpm (maybe because I have no crypto chip on my system?).

Since start-instance works OK the problem is not just that tcsd(8) is not running, but it would be interesting to know if the problem happens on an S11 system where tcsd(8) is running.

----------------- lwp# 2 / thread# 2 --------------------
fee91b55 connect (100, fe2ed1c0, 10, 1)
fed9ca03 connect (100, fe2ed1c0, 10, ccfaf417) + 23
ccfaf480 send_init (9c81718, 0, 9b5e1a0, ccfaf16a) + b8
ccfaf22d sendTCSDPacket (9c81718, 0, cd170a40, ccfaafb2) + d1
ccfaafe0 RPC_OpenContext_TP (9c81718, fe2ed278, fe2ed27c, fe2ed274) + 3c
ccfab12d RPC_OpenContext (c0000004, 81bf330, 1, ccfaa93e) + 4d
ccfaa9d1 Tspi_Context_Connect (c0000004, 0, fe2ed31c, cd01de0a) + a1
cd01de34 open_tss_context (fe2ed35c, fee920f5, cd033978, cd01deae) + 38
cd01dec0 token_specific_init (0, 0, fe2ed35c, cd014267) + 20
cd0144b9 ST_Initialize (cd036748) + 33d
cd0093dc C_Initialize (927a610, cd1d8968, fe2ed3e8, cd1d6615) + 158
cd1d6830 pkcs11_slot_mapping (9c32008, 927a610, fe2ed498, cd1d1587) + 22c
cd1d15b3 C_Initialize (927a610) + fb
cd21629a Java_sun_security_pkcs11_wrapper_PKCS11_C_1Initialize (8072d18, fe2ed518, fe2ed514, 100, f7214d18, fe2ed4e4) + 4e
faa0a152 ???????? (0, faa080f9, f71d0ab0, f7214d18, 1, cef09e88)
faa0308d ???????? (0, f7214d18, 0, f71d0ab0, ceef2570, f714c578)
faa02f27 ???????? (0, 0, 0, 0, 0, 0)
faa0308d ???????? (da9e03d8, f70f4640, 8073438, 1fa0, febc8000, 2)
faa00348 ???????? (fe2ed650, fe2ed95c, a, ceee1e98, faa090e0, fe2ed8d0)
fe50fe95 _1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThreadv (fe2ed958, fe2ed728, fe2ed8cc, 8072c00) + 1c9
fe510137 _1cCosUos_exception_wrapper6FpFpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThreadv2468_v (fe50fccc, fe2ed958, fe2ed728, fe2ed8cc, 8072c00) + 27
fe51016f _1cJJavaCallsEcall6FpnJJavaValue_nMmethodHandle_pnRJavaCallArguments_pnGThreadv (fe2ed958, 8073438, fe2ed8cc, 8072c00) + 2f
fe9b776a _1cKReflectionGinvoke6FnTinstanceKlassHandle_nMmethodHandle_nGHandle_bnOobjArrayHandle_nJBasicType_4bpnGThreadpnHoopDesc_ (8073434, 8073438, 8073440, 0, 8073430, e) + c3e
fe562060 _1cKReflectionSinvoke_constructor6FpnHoopDesc_nOobjArrayHandle_pnGThread2 (f70e29f8, 807342c, 8072c00) + 200
fe561c50 JVM_NewInstanceFromConstructor (8072d18, fe2edbac, fe2edba8) + 118
fe21ccbf Java_sun_reflect_NativeConstructorAccessorImpl_newInstance0 (8072d18, fe2edba0, fe2edbac, fe2edba8, f70e2a60, f70e2a48) + 23
faa0a152 ???????? (ce6d1280, faa08116, f70e2a38, f70e29f8, fe2edbb0, ce6d0f88)
faa02f27 ???????? (0, f70e2a38, f70e2a48, fe2edbe4, ce6d167d, fe2edc10)
faa02f27 ???????? (f70e2a38, f70e2a60, fe2edc14, ce646024, fe2edc44, ce6cfaf0)
faa03403 ???????? (0, f70e2a38, f70e29f8, fe2edc48, ce90c3b7, fe2edc80)
faa02f27 ???????? (f70e29f8, 0, ceee5210, da6007f8, f70338b8, 8072c00)
faa00348 ???????? (fe2edcd0, fe2eded0, a, ce90c490, faa090e0, fe2ededc)
fe50fe95 _1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThreadv (fe2edecc, fe2edda8, fe2eded8, 8072c00) + 1c9
fe510137 _1cCosUos_exception_wrapper6FpFpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThreadv2468_v (fe50fccc, fe2edecc, fe2edda8, fe2eded8, 8072c00) + 27
fe51016f _1cJJavaCallsEcall6FpnJJavaValue_nMmethodHandle_pnRJavaCallArguments_pnGThreadv (fe2edecc, 8073428, fe2eded8, 8072c00) + 2f
fe54eaca JVM_DoPrivileged (8072d18, fe2ee0b8, fe2ee0c0, 0, 0) + 34a
fe219f98 Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedAction_2 (8072d18, fe2ee0b8, fe2ee0c0, f70338b8, fe2ee090, 0) + 28
faa0a152 ???????? (ce685ce8, faa08116, f70338b8, fe2ee0c4, ce8cde48, fe2ee0f0)
faa02f27 ???????? (0, da9e03a0, 6fb666b1, da6007f8, fe2ee0f4, ce8cdd75)
faa02f27 ???????? (0, 0, 0, da6007f8, 0, da9e03a0)
faa02f27 ???????? (0, 0, da9d1170, fe2ee170, ce8d382d, fe2ee19c)
faa02f27 ???????? (0, da9e0478, fe2ee1a0, ce8d38ba, fe2ee1cc, ce8d3c08)
faa02f27 ???????? (0, da9e0478, f70338a0, 807344c, ce8c9000, fe2ee1fc)
faa9bcb8 ???????? (fec0fd68, 10, 28, ce689750, ce68e148, ce6064d8)
ce600150 ???????? ()

Comment by Byron Nevins [ 14/Jan/11 ]

Nothing has yet worked.

Comment by Byron Nevins [ 14/Jan/11 ]

This fixes the problem

-Dsun.security.pkcs11.enable-solaris=false

maybe not. Look into it further...

Comment by Byron Nevins [ 18/Jan/11 ]

The bug is caused by native code, Security SPI, provided by the JDK. It is too risky to resolve for 3.1

Game Plan:
We need to resolve this within 12 weeks of Solaris 11 introduction.
For initial 3.1 release we document it.

Comment by Joe Di Pol [ 18/Jan/11 ]

More data:

I ran "truss -f -t connect -v connect -p 13064", where 13064 was the pid of the instance I was going to restart. Here is the interesting part:

When things work fine I see:

13076/2: connect(50, 0xFE12D2A0, 16, SOV_DEFAULT) Err#146 ECONNREFUSED
13076/2: AF_INET name = 127.0.0.1 port = 30003
13076/2: connect(50, 0xFE12D220, 16, SOV_DEFAULT) Err#146 ECONNREFUSED
13076/2: AF_INET name = 127.0.0.1 port = 30003
13076/2: connect(50, 0xFE12D2C0, 16, SOV_DEFAULT) Err#146 ECONNREFUSED
13076/2: AF_INET name = 127.0.0.1 port = 30003

When I get the hang I see:

13078/14: connect(280, 0xCD1906A0, 16, SOV_DEFAULT) (sleeping...)
13078/14: AF_INET name = 127.0.0.1 port = 30003
13076/2: Received signal #13, SIGPIPE [caught]
. . .
13076/2: Received signal #13, SIGPIPE [caught]
13078/14: connect(280, 0xCD1906A0, 16, SOV_DEFAULT) Err#145 ETIMEDOUT
13078/14: AF_INET name = 127.0.0.1 port = 30003
13076/2: Received signal #13, SIGPIPE [caught]
13078/14: connect(280, 0xCD190620, 16, SOV_DEFAULT) (sleeping...)
13078/14: AF_INET name = 127.0.0.1 port = 30003
13076/2: Received signal #13, SIGPIPE [caught]
13076/2: Received signal #13, SIGPIPE [caught]
13078/14: connect(280, 0xCD190620, 16, SOV_DEFAULT) Err#145 ETIMEDOUT
13078/14: AF_INET name = 127.0.0.1 port = 30003
13078/14: connect(280, 0xCD1906C0, 16, SOV_DEFAULT) Err#146 ECONNREFUSED
13078/14: AF_INET name = 127.0.0.1 port = 30003

Observations:

  • The connect is trying to connect to port 30003. That's the default tcsd(8) port.
  • When connect is hanging it fails with ETIMEDOUT instead of failing immediately with ECONNREFUSED.
  • It looks like the connect in question is called 3 times.
  • During a "hang" netstat -a shows:
    localhost.59145 localhost.30003 0 0 131072 0 SYN_SENT
  • Netstat shows nothing listening on port 30003
Comment by Joe Di Pol [ 18/Jan/11 ]

sendTCSDPacket() and send_init() are in the Trouser 0.3.4 client code (0.3.4 looks to be the version on my Solaris 11 box). The Trouser 0.3.4 source can be found at: http://sourceforge.net/projects/trousers/files/trousers/0.3.4/. I've attached rpc.c that contains sendTCSDPacket() and send_init(). Nothing seems obviously wrong. Note in particular that send_init() does close the socket on an error.

Comment by Byron Nevins [ 24/Jan/11 ]

In 3.1 this will be documented.
It should be fixed for 3.2

Comment by Paul Davies [ 16/May/11 ]

The Release Notes should just mention the problem of intermittent slow start on Solaris 11. There is no workaround (except, perhaps, to wait).

Comment by Scott Fordin [ 16/May/11 ]

Added issue to 3.1.1 Release Notes.





[GLASSFISH-15217] create-service: cross-platform inconsistencies in response message Created: 15/Dec/10  Updated: 15/Dec/10

Status: Open
Project: glassfish
Component/s: None
Affects Version/s: 3.1_ms07
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Paul Davies Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The location of the log file as stated in the message from this subcommand is inconsistent between platforms.

On Windows, the message states:

This message is also available in a file named PlatformServices.log in the domain's root directory

On Linux, the message states:

For your convenience this message has also been saved to this file: /export/glassfish3/glassfish/domains/domain1/PlatformServices.log






[GLASSFISH-6780] Monitoring Tray Icon Created: 15/Nov/08  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: V3
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: mkarg Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 6,780
Tags: 3_1-exclude

 Description   

Lots of "native" Windows Services, like "Exchange" or "MS SQL Server" supply a
tray icon to the admin's desktop that allows to (remotely) see the "health
state" of the service. This is really nice since a tray icon is always there and
it can call for help in a simple way. So I want to suggest that GlassFish comes
with such a tray icon.

  • Shows the "health state" of the service: Could check several paramters (the
    service's "blood pressure" and "pulse"), so the admin can see whether the
    service is idle, running at it's limits, hanging or crashed, or just stopped by
    intention.
  • Shows the most essential health state information in the tool tip when the
    mouse hovers over the icon.
  • Shows a "bubble" automatically, containing alerts, when bad things have
    happened (like the server not responding anymore, IP connections broken, or
    another admin stopped the instance).
  • Has popup menu containing the most essentials commands, like "Administrate"
    (opens the browser to http://xyz:4848), "Shut Down", "Start Up".

This would be a really smart tool for the most essential operational monitoring.



 Comments   
Comment by mkarg [ 27/Apr/09 ]

This is not a bug but a feature request, so I changed the issue type from DEFECT
to FEATURE.

Comment by Tom Mueller [ 06/Jan/11 ]

User is no longer on the project. Reassigning to default assignee for monitoring.

Comment by Byron Nevins [ 07/Feb/11 ]

I'm guessing that the other platforms have something similar as well.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-7033] "asadmin monitor": Add capability to limit output fields Created: 13/Jan/09  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: V3
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: rpetruzzelli Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 7,033
Tags: 3_1-exclude

 Description   

ref: "./asadmin monitor --type webmodule --interval 5 --filter //server/someurl
--user admin --passwordfile ./.pwd server"

Provide a way to limit the fields displayed when running "asadmin monitor..."
For example, Omit the "ss" field. When there are many sessions the ss listing is
too overwhelming.

--fields=asc,ash,est,jc,jec,jrc,rst,svpt,ss,sst
or maybe just:
--omitfields=ss



 Comments   
Comment by Tom Mueller [ 06/Jan/11 ]

User is no longer on the project. Reassigning to default assignee for monitoring.

Comment by Byron Nevins [ 07/Feb/11 ]

We need many changes like this for the monitor command

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-3927] MBean ClassLoader Enhancements Created: 17/Dec/07  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: v2.1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: binod Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Sun


Issuezilla Id: 3,927
Tags: 3_1-exclude

 Description   

Currently jars containing the custom mbeans need to be dropped in the
AS_HOME/lib directory. Normally we expect the users to drop the libraries to
domains/domain1/lib directory.

It will be good, if MBeanClassLoader is enhanced to look at domains/domain1/lib
directory also.

Additionally, it will also be good, if the custom mbeans can be in the
application classloader.



 Comments   
Comment by kumara [ 01/Sep/09 ]

Changing version from 9.1.1 to v2.1 to reflect new name/version.

Comment by Tom Mueller [ 27/Jul/10 ]

This appears to be a monitoring issue.

Comment by Tom Mueller [ 25/Jan/11 ]

Cleared the Fix version since this issue isn't going to be fixed in 9.1.1.

Comment by Byron Nevins [ 07/Feb/11 ]

Custom Mbeans is Admin – not monitoring.

I don't know if we still support them. Keeping this issue alive uintil I have time to investigate.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-4810] Passing Java options to v3 thru startserv Created: 17/Apr/08  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: V3
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: vivekp Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Linux
URL: https://glassfish.dev.java.net/servlets/ReadMsg?list=dev&msgNo=6720


Issuezilla Id: 4,810

 Description   

I was looking at easy way to pass system properties or java options while using
v3 startup script 'startserv'.

Can we not have JAVA_OPTS env variable to pass such values:

11c11
< exec java $JAVA_OPTS -jar "$AS_INSTALL_LIB/admin-cli-10.0-SNAPSHOT.jar"
start-domain --verbose "$@"

> exec java -jar "$AS_INSTALL_LIB/admin-cli-10.0-SNAPSHOT.jar" start-domain
--verbose "$@"

I tried and it does not seem to work. Although if I start using
glassfish-10.0-SNAPSHOT.jar instaed of admin-cli-10.0-SNAPSHOT.jar then the
system properties passed to java works! I could have made it work using java
-jar from command line but it is hard to automate or document as the jar file
name will keep changing across different v3 milestones and providing options to
the startup script to pass such otions to java is available in most of the web
servers such as tomcat (catalina.sh).



 Comments   
Comment by Byron Nevins [ 17/Apr/08 ]

.

Comment by km [ 17/Apr/08 ]

Can we talk about this before you implement?

Thanks.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-4279] asadmin commands: Runtime Management: Call Flow Created: 27/Feb/08  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: command_line_interface
Affects Version/s: V3
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 4,279
Status Whiteboard:

v3-prd-item

Tags: 3_1-exclude

 Description   

Add the following asadmin commands for Call Flow:

start-callflow-monitoring
stop-callflow-monitoring



 Comments   
Comment by Jennifer Chou [ 27/Feb/08 ]

change to Feature

Comment by msreddy [ 29/Feb/08 ]

v3-prd-item

Comment by msreddy [ 29/Feb/08 ]

->harpreet

Comment by Tom Mueller [ 29/Jun/10 ]

Reassign

Comment by Bill Shannon [ 29/Jun/10 ]

My understanding is that we don't have callflow monitoring in 3.1.
I don't know whether we will have it for 3.2.

Reassigning to Nazrul, who can either close it, mark it as "target 3.2",
or assign it to someone who is working on this.

Comment by Tom Mueller [ 25/Jan/11 ]

Cleared the Fix version field since this issue isn't going to be fixed in V3.

Comment by Byron Nevins [ 07/Feb/11 ]

Idea for 3.2 improvement

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-5379] asadmin in monitoring mode creates new connections with --iterations Created: 25/Jul/08  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 9.1peur2
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: binujohn Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Sun


Issuezilla Id: 5,379
Tags: 3_1-exclude

 Description   

During some tests with the monitoring framework using asadmin, I noticed that a
new connection is being created for each request even when --iterations flag is
set. Here is a concrete example.

I was monitoring the number of connections in the keep alive state using the
following command -

#./asadmin get -m --iterations 100 --interval 5
server.http-svice.connection-queue.countqueued-count
server.http-service.keep-alive.countconnections-count
server.http-service.keep-alive.counthits-count
server.http-service.keep-alive.counttimeouts-count

server.http-service.connection-queue.countqueued-count = 0
server.http-service.keep-alive.countconnections-count = 62
server.http-service.keep-alive.counthits-count = 298
server.http-service.keep-alive.counttimeouts-count = 0

server.http-service.connection-queue.countqueued-count = 0
server.http-service.keep-alive.countconnections-count = 63
server.http-service.keep-alive.counthits-count = 299
server.http-service.keep-alive.counttimeouts-count = 0

As you can see from the 'countconnections-count' value, every iteration is
creating a new connection. I verified this using netstat as well.

asadmin should reuse one connection in the --iterations mode, rather than create
a new connection for each request.

There are a couple of reasons for requesting this change -
1. Creating a new connection per request is inefficient and this will put more
load on the server that is being monitored.
2. The monitoring itself will skew the data that is being measured (for example
changing the interval will increase the countconnections-count). We should try
to keep the intrusive effect of monitoring as low as possible.



 Comments   
Comment by sanandal [ 11/Jan/09 ]

"Reclassifying as P4 because this issue is not deemed "must fix" for this v2.1
release whose primary release driver is SailFin.
This issue will be scrubbed after this release and will be given the right
priority for the next release."

Comment by Tom Mueller [ 06/Jan/11 ]

User is no longer on the project. Reassigning to default assignee for monitoring.

Comment by Byron Nevins [ 07/Feb/11 ]

The "iterations" option does not exist in V3+ so this does not apply as a bug.

I'm leaving this issue alive as a new feature for 3.2

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-5165] asadmin monitor should provide a logging or batch mode Created: 16/Jun/08  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 9.1peur2
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: writtmeyer Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 5,165
Tags: 3_1-exclude

 Description   

I want to use "asadmin monitor" in some scripts to periodically send me some
reports. But right now there is no logging/batch mode (like e.g. "top -l 1" or
"top -n 1", depending on which top version you are using).

For a command used to monitor something I think this is essential.



 Comments   
Comment by writtmeyer [ 26/Oct/09 ]

This issue has been filed originally for v2.1. But it is still relevant for v3
(tested with promoted build b69). Thus I've changed the version to v3.

Retested as part of the FishCAT program.

Comment by janey [ 28/Feb/10 ]

reassign to bill.

Comment by Bill Shannon [ 01/Mar/10 ]

Reassign.

Comment by Tom Mueller [ 06/Jan/11 ]

User is no longer on the project. Reassigning to default assignee for monitoring.

Comment by Byron Nevins [ 07/Feb/11 ]

An improvement idea.

Perhaps the scripting client (in OGS) is what you want?

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-18234] Restart required not cleared with instance restart Created: 21/Jan/12  Updated: 26/Jan/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1.2_b18
Fix Version/s: None

Type: Bug Priority: Major
Reporter: lidiam Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

ogs-3.1.2-b18.zip


Attachments: Text File server.log.txt    
Tags: 312_qa, 312_regression, 3_1_2-exclude

 Description   

Restart required is not cleared with instance restart but it is cleared when executing stop and then start instance. Steps to reproduce:

1. Follow steps from issue 18233 to get a clustered instance in restart required state, since it's inconsistent right now.
2. Go to clustered instance page and click on Restart button. After server restarts, the instance is still displayed as requiring restart.
3. Click on Stop and then Start buttons. Now the instance is reported as running.

This does not work in CLI either:

  1. asadmin list-instances
    Enter admin password for user "admin">
    cll01 running; requires restart
    clj01 running
    Command list-instances executed successfully.
  2. asadmin restart-instance cll01
    Enter admin password for user "admin">
    cll01 was restarted.
    Command restart-instance executed successfully.
  3. asadmin list-instances
    Enter admin password for user "admin">
    cll01 running; requires restart
    clj01 running
    Command list-instances executed successfully.

There are no errors in server.log



 Comments   
Comment by Tom Mueller [ 25/Jan/12 ]

The cause of this is that synchronization is not working with restart-instance. The restart-instance command does not pass a limited use token to the _restart-instance command that runs on the instance, so it does not pass it to start-local-instance, so start-local-instance is unable to authenticate with the DAS to do resynchronization. This means that the restart required flag is not cleared on the DAS when the instance is restarted.

Comment by Tom Mueller [ 26/Jan/12 ]

Byron, please take a look at whether there would be a simple fix for this for 3.1.2.
If not, we'll defer this for 3.1.2 and fix it on the trunk only.

Comment by Joe Di Pol [ 26/Jan/12 ]

Since this isn't a 3.1.2 stopper I'm excluding it. If we happen to quickly come up with a simple, low risk fix we can revisit this for 3.1.2.

Comment by Byron Nevins [ 26/Jan/12 ]

1-2 days work to implement a solution.
1 day for thorough testing and adding devtests.
Total = 3 days

Notes -

see NodeRunner.java ~~ line 150 to see how to make a token
JavaClassRunner in common-util needs to be expanded to allow writng lines to stdin
restartinstance.java needs tocall restartrestartinstance with the magic temp. certificate.
restartrestartinstance then calls JavaClassRunner's new ctor (see below) with the "stdinlines"

Why is this hairy?

  • Have to handle the case where the instance was originally started with --_auxinput. I.e. can't just blindly add the follo9wing to the process command line:
    "--_auxinput -"
  • 90% of the code is handling NON-ASADMIN ways of starting:
    java -jar glassfish.jar
    etc., etc.
    Is the auxinput going to work for these non-asadmin calls? Needs plenty of testing.

============

Comment by Byron Nevins [ 26/Jan/12 ]

Definitely exclude this from 3.1.2

Comment by Byron Nevins [ 26/Jan/12 ]

The question was asked:

"Why doesn't restart-instance just do stop-instance followed by start-instance"?

1) Feature – we remember to restart with exactly the same possibly long & complicated list of arguments. A plain start could not do that.

2) if the instance is running in verbose mode, the output window is elegantly re-used (try it). It would be impossible with a plain start at least on Windows.

3) you can start many different (non-asadmin) ways and it will work.

4) This is the big one. If the instance is remote and is using a config node (i.e. no SSH, no DCOM) then once you stop the instance – that's it. You can never start it again from DAS. User has to go to the remote.
THIS IS A HUGE FEATURE!!!

Comment by Byron Nevins [ 26/Jan/12 ]

One final note. When restart was implemented there was no such temporary security token. When that code was added for start-xxx, restart was not included in the change!

Comment by Tom Mueller [ 26/Jan/12 ]

To reproduce:

1. start the domain, change the admin password to something, create a node on another host, create an instance i1

2. start the instance

3. do something to cause the restart-required flag to be set, such as:
asadmin set-log-attributes --target i1-config com.sun.enterprise.server.logging.GFFileHandler.rotationTimelimitInMinutes=3

4. restart the instance:

asadmin restart-instance i1

5. look at the status of the instances:

asadmin list-instance -l

Observer that the restart-required flag is still set on i1.

If you turn on the admin logging (javax.enterprise.system.tools.admin.level=FINE) then you will see log messages in the log that show that a user was not able to login from the host that is running the instance. These are the _synchronize-instance commands that are failing.





[GLASSFISH-18206] Endless Network Timeout Created: 18/Jan/12  Updated: 18/Feb/13

Status: Open
Project: glassfish
Component/s: distributed management
Affects Version/s: 3.1.2_b17, 4.0_b19
Fix Version/s: future release

Type: Bug Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3_1_2-exclude

 Description   

GF Admin clustering tasks are far too subject to ridiculously long network timneouts. E.g. in this bug we wait a full 10 minutes to get output from "asadmin version".

How to reproduce:

0. Remote windows box has glassfish installed.
1. validate-dcom works fine from (different) DAS machine
2. Sabotage asadmin.bat on the remote machine so that it hangs. Easy! See [1] below
3. Note how the command hangs for a very very long time.

[1] Add this to asadmin.bat's java call
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=1234

=========================
How did I find this out? Adding the line in[1] is an essential trick for debugging remote GF calls. I forgot about it and left it in.






[GLASSFISH-18185] 'update-node-ssh' command hangs when ssh port is not provided. Created: 13/Jan/12  Updated: 25/Jan/12

Status: Open
Project: glassfish
Component/s: distributed management
Affects Version/s: 3.1.2_b17
Fix Version/s: None

Type: Bug Priority: Major
Reporter: lidiam Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

ogs-3.1.2-b17.zip


Attachments: JPEG File dcom-to-ssh-error.JPG     Text File server.log.txt    
Issue Links:
Related
is related to GLASSFISH-18209 Endless SSH Network Timeout Open
Tags: 312_gui_new, 312_qa, 3_1_2-exclude, 3_1_2-release-note-added, 3_1_2-release-notes

 Description   

Currently it's not possible to convert a DCOM node to an SSH node in Admin Console. For an existing DCOM node, when user changes to SSH (and selects password authentication in my case), and hits Save, the long running process popup is there for a long time, about 15 minutes. Then the following error is displayed:

An error has occurred
Check server log for more information.

The server.log file contains:

[#|2012-01-12T17:46:55.953-0800|WARNING|glassfish3.1.2|javax.enterprise.system.t
ools.admin.com.sun.enterprise.v3.admin.cluster|_ThreadID=97;_ThreadName=Thread-2
;|Could not connect to host jed-asqe-43 using SSH.: There was a problem while co
nnecting to jed-asqe-43:135: Operation interrupted: host=jed-asqe-43 port=135 us
er=j2eetest password=<concealed> keyFile=/export/home/j2eetest/.ssh/id_rsa keyPa
ssPhrase=<concealed> authType=null knownHostFile=/export/home/j2eetest/.ssh/know
n_hosts|#]

[#|2012-01-12T17:46:55.953-0800|SEVERE|glassfish3.1.2|org.glassfish.admingui|_Th
readID=98;_ThreadName=Thread-2;|java.io.InterruptedIOException: Operation interr
upted;
java.io.InterruptedIOException: Operation interrupted;
restRequest: endpoint=https://localhost:4848/management/domain/nodes/node/jedy/u
pdate-node-ssh
attrs={sshpassword=*******, installdir=C:\as\dcomtest\glassfish3, nodehost=jed-a
sqe-43, sshuser=${user.name}}
method=POST|#]

The conversion works in CLI with the following command:

asadmin update-node-ssh --nodehost <host name> --sshport 22 --sshuser <user name> <node name>

However, if I execute the following in CLI, to convert DCOM to SSH node, it also hangs and then fails:

asadmin update-node-ssh --nodehost <host name> <node name>

Thus my guess is that Admin Console does not pass along the other two options (that should not be required). Assigning to Admin Console to verify and pass on to CLI, if this is the case. I understand this will most likely not get fixed for this release.



 Comments   
Comment by Anissa Lam [ 13/Jan/12 ]

As shown in the log Lidia pasted, the REST request sent is:

restRequest: endpoint=https://localhost:4848/management/domain/nodes/node/jedy/update-node-ssh
attrs={sshpassword=*******, installdir=C:\as\dcomtest\glassfish3, nodehost=jed-asqe-43, sshuser=${user.name}}
method=POST|#]
so, console is sending in all the info that user enters. At that time, Lidia probably didn't set the ssh-port. I verified that if sshport is specified, it will be sent in as well. I believed console is doing the correct thing.

Transfer to backend for evaluation.

Comment by lidiam [ 13/Jan/12 ]

That's correct, I did not set the ssh port since the default is always set to 22 and there was no need to change that.

Interesting thing to note is that if I create a new SSH node ssh port field is prepopulated to 22. If I choose to convert CONFIG node in Admin Console, ssh port is populated with 22, however, when I choose to convert DCOM node to SSH, ssh port field is not populated, so there is an inconsistency here. In fact if I enter ssh port in Admin Console when converting DCOM node to SSH, it works fine - we can document it as a workaround.

There are two issues here then:

1. ssh port is not populated when switching from DCOM to SSH node.
2. update-node-ssh command hangs when ssh port is not provided.

Comment by Anissa Lam [ 13/Jan/12 ]

I will file a separate P4 bug about sshport not populated when converting from DCOM to SSH.

I changed the summary of this bug to correctly reflect the issue.

This affects both CLI and GUI. I don't know how often user will convert DCOM node to SSH node, but if we decide to release note this, the work around is

  • ensure 'sshport' param is specified if using CLI
  • fill in port number (default is 22) when doing that in GUI.
Comment by Byron Nevins [ 18/Jan/12 ]

The problem is that a DCOM node has the "sshport" set to 135. When you run update-node-ssh it can't tell apart these 2 scenarios:

1) You're running the command on an existing ssh node that happens to use port 135 instead of 2
2) You're converting from a DCOM node that always has 135 set as the port number.

============

This is a gray area. Technically the software did precisely what you asked it to do – it updated the node config with the data you gave it and only the data you gave it. That's how it was designed.

It should be fixed in 4.0.

Recommended Fix

DCOM shouldn't bother with the port setting of 135. That's in a much lower abstraction - we never have to deal with the port number. DCOM should simply never use the sshport field.





[GLASSFISH-17886] nucleus contains javaee monitoring related code Created: 02/Dec/11  Updated: 21/Sep/15

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 4.0
Fix Version/s: 4.1.1

Type: Bug Priority: Major
Reporter: Sanjeeb Sahoo Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: nucleus-cleanup, spo

 Description   

see files in trunk/nucleus/common/common-util/src/main/java/com/sun/enterprise/admin/monitor/callflow/



 Comments   
Comment by Sanjeeb Sahoo [ 07/Apr/12 ]

any update on when this will be fixed?





[GLASSFISH-17520] restart-instance and stop-local-instance have inconsistent views of whether the instance is up causing devtest failure Created: 28/Oct/11  Updated: 21/Sep/15

Status: In Progress
Project: glassfish
Component/s: admin
Affects Version/s: 3.1.2_b06, 4.0
Fix Version/s: 4.1.1

Type: Bug Priority: Major
Reporter: Tom Mueller Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3_1_2-exclude

 Description   

The admin devtests on the trunk and the 3.1.2 branch are current failing due to an issue with instance runtime status checking in the stop/start/restart commands. This problem has been there for sometime, but it was recently made visible by an change that cause the REST service to be started as part of instance startup.

Here is the problem. During instance startup, there is the following sequence of events:

1) AdminAdapter starts (which allows admin commands to be executed)
2) other services start (REST is now one of these)
3) the pid file is written.

The restart-domain command calls _get-runtime-info to determine if the server has started. So restart-domain thinks the server is started when (1) is completed.

The stop-local-instance command looks for the pid file. So it thinks the server is started when (3) is completed.

If (2) takes a relatively long time, it is possible for restart-domain to complete successfully, and then stop-local-instance to run immediately after it and think that the just started instance is still down. So stop-local-instance doesn't actually stop the instance.

To reproduce this problem, run this script on an instance:

asadmin restart-instance i1
asadmin restart-instance i1
asadmin stop-local-instance i1

It has to be in a script so that the commands are executed without a delay.

To fix this, the various commands need to be given a consistent view of when the server is up. One possibility is that _get-runtime-info needs to check if the pid file has been written before returning the pid.



 Comments   
Comment by scatari [ 07/Dec/11 ]

Please correct me if I am wrong, I do not see a customer use case here that has a quick stop followed by a restart. Does it need to be considered for 3.1.2?

Comment by Byron Nevins [ 14/Dec/11 ]

Can't reproduce.
Not a likely scenario for users.

Should investigate in 4.0
Should not fix for 3.1.2

Also not reproducible:

d:\gf\branches\3.1.2\admin>call asadmin restart-instance i1
i1 was restarted.
Command restart-instance executed successfully.
i1 was restarted.
Command restart-instance executed successfully.
The instance, i1, is stopped.
Command stop-instance executed successfully.

Comment by Byron Nevins [ 18/Feb/13 ]

confirmed. Easy to reproduce with the sample 3 command script...





[GLASSFISH-17911] update-node-com error message refers to SSH Created: 06/Dec/11  Updated: 06/Mar/12

Status: Open
Project: glassfish
Component/s: distributed management
Affects Version/s: None
Fix Version/s: not determined

Type: Bug Priority: Major
Reporter: Paul Davies Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3_1_2-exclude

 Description   

An unsuccessful attempt to update a DCOM node displays an error message that refers to SSH:

asadmin> update-node-dcom -w hudson --nodehost  host.example.com xkyd
remote failure: Warning: some parameters appear to be invalid.
SSH node not updated. To force an update of the node with these parameters rerun
the command using the --force option.
com.sun.enterprise.universal.process.WindowsException: org.jinterop.dcom.common.
JIException: Access is denied, please check whether the [domain-username-password]
are correct. Also, if not already done please check the GETTING STARTED and
FAQ sections in readme.htm. They provide information on how to correctly configure
the Windows machine for DCOM access, so as to avoid such exceptions.  [0x00000005]
Command update-node-dcom failed.


 Comments   
Comment by Joe Di Pol [ 30/Jan/12 ]

Not a 3.1.2 stopper.

Comment by Tom Mueller [ 06/Mar/12 ]

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.





[GLASSFISH-17553] convert install-node and uninstall-node (dcom, ssh) to remote command Created: 01/Nov/11  Updated: 30/Nov/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Anissa Lam Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Currently, the following commands are local command. GUI needs to be a remote command so we can add the feature in GUI.
install-node, install-node-ssh, install-node-dcom and the corresponding uninstall command.






[GLASSFISH-17258] Determine Default Name of Probes Automatically Created: 30/Aug/11  Updated: 30/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

@Probe
//@Probe(name = "getMovieStarted")
public void getMovieStarted(@ProbeParam("nano") long nano) {
}

=========

I tried the above code and discovered that Flashlight accepts it – but it sets the name of the probe to null!!:

JavaOne:19120-app:MovieSessionBean: (long nano)

instead of the desired:
JavaOne:19120-app:MovieSessionBean:getMovieStarted(long nano)






[GLASSFISH-17182] Make monitoring framework completely extensible without framework changes Created: 10/Aug/11  Updated: 10/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on GLASSFISH-17085 Provide way to write monitoring liste... Open
depends on GLASSFISH-17166 Remove hard-coded container-specific ... Open
depends on GLASSFISH-17167 Make the monitor command generic Open

 Description   

This is an umbrella bug for removing glassfish-specific code from monitoring so monitoring can be easily extended:

GLASSFISH-17166 Remove hard-coded container-specific monitoring configuration from ModuleMonitoringLevels config bean
GLASSFISH-17167 Make the monitor command generic
GLASSFISH-17085 Provide way to write monitoring listener without modifying framework classes






[GLASSFISH-17176] New annotation @TimedProbe to probe the beginning and end of a method and calculate total time Created: 09/Aug/11  Updated: 09/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Commonly needed is how much time is spent in a method. Create a new @TimedProbe which does this automatically.






[GLASSFISH-17175] Add begin and end attributes to @Probe to instrument either at the beginning or end of the method. Created: 09/Aug/11  Updated: 09/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Currently probes are only instrumented at either the beginning (or end - not sure which) of the method.
Adding begin and end attributes allows more flexibility, esp. usefully in calculating processing times.



 Comments   
Comment by Byron Nevins [ 09/Aug/11 ]

The injected code is put at the BEGINNING of the method

You can verify by:

1) set AS_DEBUG=true in your environment
2) decompile and look at any of the re-written .class files in (on my machine)
D:\glassfish3\glassfish\flashlight-generated
– of course you have to look at an instrumented method that actually *has* some code!

Example 1:

public static void gc()

{ ProbeRegistry.invokeProbe(177, new Object[0]); Runtime.getRuntime().gc(); }

=================

Example 2
Deploy MonApp [1] and access it in a browser first

@Probe(name="myProbe")
public void myProbe(String arg1)
{
ProbeRegistry.invokeProbe(157, new Object[]

{ ??? }

);
String s;
System.out.println("inside myProbeMethod called with this arg: " + s);
}

[1]
Source: https://svn.java.net/svn/glassfish~svn/trunk/v2/appserv-tests/devtests/admin/cli/apps/MonApp
War file: https://svn.java.net/svn/glassfish~svn/trunk/v2/appserv-tests/devtests/admin/cli/resources/MonApp.war





[GLASSFISH-17172] Improving Monitoring Tree Created: 09/Aug/11  Updated: 09/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Investigate improving monitoring tree for stability (There were many regressions in 3.1).

Potentially refactor AbstractTreeNode:

  • Remove if possible "band-aid" fixes:
  • encoding/decoding
  • Better way to do pattern matching?
  • Currently, in getNodes(String pattern) the dotted name pattern is converted to regex. Code goes through the entire list of tree nodes and compares the pattern with the complete dotted name of the nodes to find the list of matching TreeNodes.

Notes:

  • A TreeNode consists of node name and whether the node is enabled. The TreeNode can also contain a method on an instance to invoke to return a Statistic object.
  • asadmin get -m (MonitoringReporter) and REST (MonitoringResource) both use the monitoring tree to display the statistics. REST uses getNode (String completeName) and get -m command uses getNodes(String pattern). REST monitoring does not take '*' wildcard like the get -m command can use. That's why they use different path to retrieve the statistics.





[GLASSFISH-17168] Improve use of '/' and '.' for the monitoring tree nodes Created: 09/Aug/11  Updated: 09/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

1)
Replace the subTreeRoot delimiter of '/' to something else: '.' is nice because it is used already in dotted names. Or use space, tab, newline, carriage return: " \t\n\r\f"
Compatibility issue?

2)
Allow tree node to contain '/' in the node name. No need for user to use _SLASH_. Example: register("web", "aaa.bbb.cc/d", statsProvider)

3)
Allow tree node to contain a '.' in the node name by using backlash to escape the '.'.
Example: register("web", "aaa.bbb.cc\.d", statsProvider)






[GLASSFISH-17167] Make the monitor command generic Created: 09/Aug/11  Updated: 10/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks GLASSFISH-17182 Make monitoring framework completely ... Open

 Description   

Current monitor command is glassfish-specific with hard-coded display headers and details for jvm, httplistener, webcontainer, servlet. Make the monitor command generic, exensible.






[GLASSFISH-17166] Remove hard-coded container-specific monitoring configuration from ModuleMonitoringLevels config bean Created: 09/Aug/11  Updated: 10/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks GLASSFISH-17182 Make monitoring framework completely ... Open

 Description   

This is part of Nucleus Cleanup






[GLASSFISH-17164] Add request id in the probe such that we can track a specific request in the server as it flows through various containers Created: 08/Aug/11  Updated: 08/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Trace a specific request from end-to-end. This is something that WebLogic already has.
We could use ThreadLocal to for the request ID.






[GLASSFISH-17165] Improve performance impact Created: 08/Aug/11  Updated: 08/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Performance impact is currenntly ~7%.






[GLASSFISH-17089] Uninstrument probe providers when probe listener is unregistered Created: 21/Jul/11  Updated: 21/Jul/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[GLASSFISH-17088] Turn on monitoring at fine-grained levels, for specific attributes Created: 21/Jul/11  Updated: 21/Jul/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: Jennifer Chou Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

It would be nice to turn monitoring on for just the things you are interested in. Right now you can turn on monitoring at the module level. How about just for a specific attribute?

We may look at the java logging and logging.properties for ideas.
Maybe specify in monitoring.properties:
aaa.bbb.cccc = FINE
aaa.bbb = FINER






[GLASSFISH-17085] Provide way to write monitoring listener without modifying framework classes Created: 21/Jul/11  Updated: 10/Aug/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Tom Mueller Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Java Source File XYZListener.java    
Issue Links:
Dependency
blocks GLASSFISH-17182 Make monitoring framework completely ... Open

 Description   

When writing an monitoring listener by extending the ProbeProviderEventListener class, it becomes necessary to modify the framework's ProbeClientInvokerFactory class to add a create*Invoker method that increments the AtomicInteger that is contained in that class.

This RFE is for modifying the flashlight framework so that it is possible to create a ProbeProviderEventListener and add it to the system without having to modify any of the framework classes.



 Comments   
Comment by Byron Nevins [ 21/Jul/11 ]

ProbeProviderEventListener is an interface not a class. You don't extend it you implement it.

Can you give more details?
What exactly do you want?

How about:
1) Sample Code for how you have to do it now.
2) Sample Code for how you would like to do it instead.

Comment by Tom Mueller [ 21/Jul/11 ]

Please see the attached file, XYZListener.java, for sample code that implements the ProbeProviderEventListener interface. In the probeProviderAdded method, there is a call to p.addInvoker(ProbeClientInvokerFactory.createXYZInvoker(p)

The code for the createXYZInvoker method is the following:

public static ProbeClientInvoker createXYZInvoker(FlashlightProbe probe)

{ int invokerId = clientMethodIdCounter.incrementAndGet(); return new XYZListener.XYZProbeClientInvoker(invokerId, probe); }

Maybe there is a way to get an invokerId without using the clientMethodIdCounter that is a private member of ProbeClientInvokderFactory, but I didn't see one.





[GLASSFISH-18577] list command: Too much info for instances! Created: 29/Mar/12  Updated: 29/Mar/12

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File badlist.txt    

 Description   

I was investigating why weare getting Admin DevTests failures regularly with

asadmin list -m "*" (Jira14389 test)

I did the following:

create-domain
create-instance i1

    • turn on monitoring full blast **
      start-domain
      start-instance
      deploy --target i1 HelloWeb.war
      ====
      I put the output from
      asadmin list -m "*"

The list output is correct for DAS. For instance i1 there is WAY too much info. It looks like it really is showing the output from "get -m *" rather than "list -m *"






[GLASSFISH-18437] install-node --force is too slow Created: 01/Mar/12  Updated: 18/Feb/13

Status: Open
Project: glassfish
Component/s: distributed management
Affects Version/s: future release
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Tom Mueller Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The install-node --force command takes too long to remove all of the files from the remote host. It prints a "Force removing file..." message for each file that it is removing, and it appears that maybe it is doing a separate "scp" command to remove each file. It appears that it is removing the entire installation (including the domains and nodes directories, which it should not do). If so, a "rm -r" would be better than removing each file individually.

I was tempted to write this as an RFE, but the current implementation is too slow to be useful.

I observed this with an SSH node.






[GLASSFISH-18410] In the monitor logging module,the MDB can not output MethodReadyCount, LowWaterMark and HighWaterMark Created: 27/Feb/12  Updated: 27/Feb/12

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: v2.1.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: lzg5039 Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

windows



 Description   

In the monitor logging module, the following items can be output in Stateless Session Bean, but why they cannot be output in MDB?

  • MethodReadyCount
  • LowWaterMark
  • HighWaterMark

All these items are relating to the number of EJB Instances that has been pooled.

Since both Stateless Session Bean and MDB are running by using the same Instance pooling mechanism, it is believed that the above information can also be output in MDB.

Anyhow, we still like to know the reason why the above information is not output in MDB only.
And the V3 also has the same question,please tell me the answer too.






[GLASSFISH-18342] Avoid committing Native binary code in GlassFish SVN workspace Created: 08/Feb/12  Updated: 08/Feb/12

Status: Open
Project: glassfish
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: janey Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Native binary files are in GlassFish SVN trunk workspace:
main/nucleus/admin/server-mgmt/src/main/resources/lib/winsw.exe
main/nucleus/admin/server-mgmt/target/classes/lib/winsw.exe
main/nucleus/cluster/cli/src/main/resources/lib/DcomConfigurator.exe
main/nucleus/cluster/cli/target/classes/lib/DcomConfigurator.exe
main/nucleus/cluster/uc/src/main/resources/lib/DcomConfigurator.exe
main/nucleus/cluster/uc/target/classes/lib/DcomConfigurator.exe
main/nucleus/cluster/vld/target/classes/lib/DcomConfigurator.exe

We should try avoiding checking-in native binaries in the source tree.

One way to avoid this is to deploy the native binaries to Maven.
See details here: http://docs.codehaus.org/display/MAVENUSER/Using+Maven+to+manage+.NET+projects






[GLASSFISH-18469] On the local and remote hosts was used the same user, but install-node-dcom failed without -w <user_name> Created: 07/Mar/12  Updated: 15/Feb/13

Status: Open
Project: glassfish
Component/s: distributed management
Affects Version/s: 3.1.2_b21
Fix Version/s: future release

Type: Bug Priority: Major
Reporter: easarina Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

3.1.2 bui8ld 21. Window 2008 machines.

I believe that this is a regression issue. install-node-dcom failed, because was not used -w <user_name> option. See, for example:
asadmin install-node-dcom -W password1.txt bigapp-oblade-3
com.sun.enterprise.util.cluster.windows.process.WindowsException: Logon failure:
unknown user name or bad password.
Command install-node-dcom failed.

But the command: "asadmin install-node-dcom -w aroot -W password1.txt bigapp-oblade-3" was executed successfully.
On the DAS machine and on the remote host was used the same user aroot, and according to the help: "The default is the user that is running this subcommand.". I.e. in my case aroot.






[GLASSFISH-18935] Flashlight Uninstrumentation Created: 24/Jul/12  Updated: 24/Jul/12

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Add the ability to "un-instrument" classes at run-time.






[GLASSFISH-18707] delete-local-instance should NOT have a --domain parameter Created: 09/May/12  Updated: 21/Sep/15

Status: Open
Project: glassfish
Component/s: distributed management
Affects Version/s: 4.0_b38_ms2
Fix Version/s: 4.1.1

Type: Bug Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

delete-local-instance is NOT a true local command. It demands that the instance's DAS be running. If it is not running then the command fails and does nothing.

Therefore it is just busy-work for the user to specify the name of the domain since we KNOW which domain it is by the usual host and port args.

Currently this issue is only on das-branch



 Comments   
Comment by Tom Mueller [ 07/Feb/13 ]

Setting the target fix version to 4.0.1 since this is not needed for the Java EE 7 RI/SDK release.





[GLASSFISH-18703] JVM option -Djavax.management.builder.initial is not written to server.log Created: 09/May/12  Updated: 20/Sep/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: tak09 Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

GlassFish 4.0 b34
Windows 7
Java 1.6.0_26


Attachments: XML File domain.xml     Text File server.log    

 Description   

The JVM options set in domain.xml are written in server.log when GlassFish is started.
However, only the following JVM option is NOT written to the server.log. All other JVM options are written to server.log.
<jvm-options>-Djavax.management.builder.initial=com.sun.enterprise.v3.admin.AppServerMBeanServerBuilder</jvm-options>

Steps to reproduce:
1. Confirm that the JVM option is set in domain.xml. "-Djavax.management.builder.initial"
2. Start Glassfish. asadmin start-domain
3. Check server.log.
You will find all JVM options except "-Djavax.management.builder.initial" are listed.



 Comments   
Comment by zhouronghui [ 20/Sep/12 ]

Dear tak09

I think this is not a bug.
Because in the source of JvmOption.java, you can find the source that ignore the JVM option like this:

-Djavax.management.builder.initial=com.sun.enterprise.*.AppServerMBeanServerBuilder

PATH : nucleus\admin\launcher\src\main\java\com\sun\enterprise\admin\launcher

JvmOptions.java
    private void filter() {
        // there is only one forbidden sys prop now so no need yet for fancy
        // data structures to contain the one key/value

        // I have seen these 2 values:
        // com.sun.enterprise.admin.server.core.jmx.AppServerMBeanServerBuilder
        // com.sun.enterprise.ee.admin.AppServerMBeanServerBuilder

        final String key = "javax.management.builder.initial";
        final String forbiddenStart = "com.sun.enterprise";
        final String forbiddenEnd = "AppServerMBeanServerBuilder";

        String val = sysProps.get(key);

        if (val != null && val.startsWith(forbiddenStart) && val.endsWith(forbiddenEnd))
            sysProps.remove(key);

        // ~ommitted~ //
    }
Comment by zhouronghui [ 20/Sep/12 ]

Dear Tom

Would please check my description above?
If it is right, Please close this issue.

Thank you.

Comment by Tom Mueller [ 20/Sep/12 ]

Your analysis is correct. However, the default domain.xml still has this setting in it. It seems that it shouldn't be there since it gets removed anyway. Assigning to Byron to confirm. Byron, if the javax.management.builder.initial setting should not be there in the server-config JVM options setting, can you please remove it.





[GLASSFISH-16751] Allow Setting Monitoring Levels for Individual Instances in a Cluster Created: 27/May/11  Updated: 18/Oct/12

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: V3
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: ee7ri_cleanup_deferred

 Description   

Currently the monitoring levels (e.g. web-container=HIGH) are part of the monitoring-service element under the config element.
All clustered instances must share the same config.

There is no reason to not allow instances to override the monitoring levels of their cluster. We just can't currently.

We should add support for doing this. Perhaps something similar to how we allow separate instances to override port numbers?



 Comments   
Comment by Tom Mueller [ 18/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issue was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-16825] If It is Impossible to Transform Classes -- Disable Monitoring Created: 08/Jun/11  Updated: 06/Mar/12

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: not determined

Type: Bug Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3_1-next, 3_1_1-scrubbed, 3_1_x-exclude

 Description   

Very early in the process of booting up a GF Server we should check and see if it is possible to do bytecode transformation of classes. If NOT then we should :

1) if monitoring is enabled, log a SEVERE message about the problem and then disable monitoring – but just for that run. Don't persist it in domain.xml

2) if monitoring is disabled – then do nothing

In either case if the user runs "enable-monitoring" we should return a strongly worded error.



 Comments   
Comment by Tom Mueller [ 06/Mar/12 ]

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.





[GLASSFISH-16775] Should Validate listeners Created: 31/May/11  Updated: 31/May/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on GLASSFISH-16755 Monitoring Invoker class not handling... Resolved

 Description   

We ought to rigorously check listeners that are added to make sure that their method signatures match the corresponding probe signatures.

Note that I have already added code that will change a primitive in the probe to a String in the listener. See 16755



 Comments   
Comment by Byron Nevins [ 31/May/11 ]

After the fact detection is fine – but it's always best to not allow such errors in in the first place.





[GLASSFISH-15930] Need an Easier way to add new Monitoring Modules Created: 09/Feb/11  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3_1-exclude

 Description   

See 15923

I fixed 15923 (svn 45033) to handle the "deployment" monitoring module. For 3.2 we should re-engineer the code in
MonitoringConfig.java and ContainerMonitoring.java

E.g. we could simply use reflection and the JavaBean API to get the right setter methods.



 Comments   
Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-16140] server running as a service on Windows exits when user logs out (need -Xrs JVM option) Created: 03/Mar/11  Updated: 17/Oct/12

Status: Reopened
Project: glassfish
Component/s: admin
Affects Version/s: V3
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Tom Mueller Assignee: Byron Nevins
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows


Issue Links:
Dependency
blocks GLASSFISH-16311 Improve operating service (OS) integr... Open
Duplicate
duplicates GLASSFISH-5263 Platform Services and the -Xrs option Resolved
Tags: ee7ri_cleanup_deferred

 Description   

From the GlassFish forum:
http://www.java.net/forum/topic/glassfish/glassfish/glassfish-v31-rc-and-asadminbat-oracle?force=200

If Oracle would like the asadmin create-service feature to be useful on Windows, then two things should be done to the default installation of GlassFish 3.1 to prevent the GlassFish Windows service from being shut down when the user logs out:
1) asadmin.bat's call to the java.exe needs -Xrs JVM option.
2) domain.xml needs a new <jvm-options>-Xrs</jvm-options>

Ryan



 Comments   
Comment by Tom Mueller [ 03/Mar/11 ]

Updated summary to read like a bug.

Comment by Byron Nevins [ 03/Mar/11 ]

I've blogged about this well-known issue:

http://blogs.sun.com/foo/entry/how_to_make_v3_platform

=================

We decided to keep this step for the user to do in 3.1. The main problem was that if I just plopped it in there you would not get stack traces with a KILL signal. Then users would be confused about that. Of course all known Java SE 6 implementations now have a jstack command [1] so that problem has mostly gone away.

Another problem is that there is no official delete-service command. So – if we added -Xrs to the server config as part of create-service, there would be no corresponding way to undo that.

Yet another problem is adding -Xrs to asadmin.bat The proper solution here, is to have a clone of asadmin.bat "start-service.bat" or something like that. Adding a new bat file to the installation is also a BIG deal.

So that left the only clean solution to be adding -Xrs to ALL server configs. This was decided to be unacceptable.

In 3.2 we will revisit this issue. I think we will probably add -Xrs to the config for all servers. After all it *is* an application server!

[1] it took FOREVER for Apple to implement it.

Comment by Byron Nevins [ 03/Mar/11 ]

D:\gf\v3\packager\nucleus-base>svn commit -F ..\svn-commit.tmp
Sending nucleus-base\bin\asadmin
Sending nucleus-base\bin\asadmin.bat
Sending nucleus-base\lib\templates\domain.xml
Transmitting file data ...
Committed revision 45384.

Comment by Byron Nevins [ 03/Mar/11 ]

Note that this does not just affect GF as a service. On the contrary! It is much much more likely to occur when running GF not-as-service!

I.e. most users will have GF-as-a-service run as system account – which never logs out. A non-service GF server, on the other hand, is almost always running as a user account and will immediately exit if the user logs out.

As of today – all GF servers will, by default, ignore all signals.

I added "-Xrs" to asadmin, asadmin.bat and to the 2 config elements in the default domain.xml

Comment by Byron Nevins [ 11/Mar/11 ]

I have heard negative feedback. I'll restore the code to the way it was, and keep this bug alive so it can be dealt with permanently for the next release.

===========================================

Hi, bnevins.

Adding -Xrs, kill -QUIT for getting threads dump must be ignored onUNIX
Systems.

So, Can you add the option just only Windows System?

How about adding it as variable on asenv.bat?

Thanks,

Comment by Byron Nevins [ 11/Mar/11 ]

http://blogs.sun.com/foo/entry/how_to_make_v3_platform

Comment by Byron Nevins [ 11/Mar/11 ]

Documentation:
http://download.oracle.com/docs/cd/E18930_01/html/821-2416/gglqp.html#giurf

Comment by Byron Nevins [ 22/Mar/11 ]

It's not feasible to put -Xrs oly for Windows. In order to do that I'd have to hide it in code and the user could not get rid of it. We can't have different default domain.xml files either.

One option is to add it as a jvm-option when the service is created on Windows but that's also a huge deal (requires the server(s) to be running. Yet more code for delete-service)

Comment by Byron Nevins [ 22/Mar/11 ]

reopen as RFE

Comment by Tom Mueller [ 22/Mar/11 ]

How about adding $

{Xrs} to the JVM options, and then in asenv.bat, add set Xrs=-Xrs, but don't add it to asenv.conf so that the ${Xrs}

doesn't expand to anything?

One problem with this is that the $

{Xrs}

could not be removed using delete-jvm-options because it wants every option to start with a dash.

Comment by Tom Mueller [ 17/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issues was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-16673] log message to server log when domain can't be started Created: 18/May/11  Updated: 18/May/11

Status: Open
Project: glassfish
Component/s: command_line_interface
Affects Version/s: 3.1
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Dies Koper Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

In GFv3.1, if you add a JVM option with an illegal value to the domain
and try to start it using asadmin start-domain, a clear error is logged:

Waiting for domain1 to start .
Error starting domain domain1. The server exited prematurely with exit
code 1. Before it died, it produced the following output:

Could not create the Java virtual machine. Unrecognized VM option
'MaxPermSize?=AAAA'

Command start-domain failed.

However, nothing is logged to the server.log.

I understand why from an implementation POV (the server.log is only
written to from the Java process that we just failed to start), but it
would be nice to have all messages in one place and have it logged in
the server log to.
I suppose this could be implemented by just copying the JVM error
message (in appropriate log format with timestamp, etc.) to the DAS'
server log while it is logged to the console. As the domain is not
running the file wouldn't be locked.

see GFlauncher. getDeadProcessTrace(Process sp)






[GLASSFISH-16526] Add restart-service command Created: 02/May/11  Updated: 17/Oct/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks GLASSFISH-16311 Improve operating service (OS) integr... Open
Tags: ee7ri_cleanup_deferred

 Description   

For the user's convenience, run the platform-dependent commands needed to restart a (GlassFish) service.



 Comments   
Comment by Tom Mueller [ 17/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issues was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-16523] Platform Services - Add Support for All UNIX Created: 02/May/11  Updated: 17/Oct/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks GLASSFISH-16311 Improve operating service (OS) integr... Open
Tags: ee7ri_cleanup_deferred

 Description   

We currently support all versions of Windows, all versions of Linux and Solaris-SMF.
Add support for all UNIX including non-SMF Solaris.



 Comments   
Comment by Tom Mueller [ 17/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issues was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-16522] Add Command for list-services Created: 02/May/11  Updated: 17/Oct/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks GLASSFISH-16311 Improve operating service (OS) integr... Open
Tags: ee7ri_cleanup_deferred

 Description   

Add list-services command



 Comments   
Comment by Tom Mueller [ 17/Oct/12 ]

Marking the fix version field as "future-release". This is based on an evaluation by John, Michael, and Tom WRT to the PRD for the Java EE 7 RI/SDK. This issues was deemed to not be a P1 for that release. If this is in error or there are other reasons why this RFE should be targeted for the Java EE 7 RI/SDK release, then change the fix version field back to an appropriate build.





[GLASSFISH-16012] RestartRequired: display reason for instances other than DAS Created: 16/Feb/11  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1_b43
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: lidiam Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3_1-exclude

 Description   

I cannot find an issue logged for this already, so here it goes. When DAS requires a restart, user can find out what configuration or other changes triggered restart required in Admin Console. However, it is not so for standalone and clustered instances. We should provide this information for all server instances to the users.



 Comments   
Comment by Anissa Lam [ 16/Feb/11 ]

_get-restart-required which provide the reason doesn't support target.

Usage: asadmin [asadmin-utility-options] _get-restart-required
[-why[=<why(default:false)>]] [?|--help[=<help(default:false)>]]

This needs to be supported before GUI can work on this.
Transfer to 'admin'. Please transfer back to GUI when this is ready so that we can display this to user.

Comment by Tom Mueller [ 18/Feb/11 ]

list-instances is the command for getting the status of an instances, and it includes the reasons that a restart is required. It takes a target option so that it is possible to get the status of just one instance.

However, it doesn't put the status information into properties in the report. The REST interface would need this so that the console can use the information.

Please evaluate this for possible inclusion in 3.2.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-21093] GlassFish 4 does not start with JDK 9 Created: 23/Jun/14  Updated: 26/Jun/14

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 4.1_b05
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Joe Di Pol Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 4_0_1-review

 Description   

When run under JDK 9 GlassFish fails to start up. Two issues have been identified. According to one user GlassFish will start if these two changes are made:

  1. remove JVM options PermSize and MaxPermSize from domain.xml
  2. add jre-1.9=$ {jre-1.8}

    in osgi.properties



 Comments   
Comment by Joe Di Pol [ 26/Jun/14 ]

Updated osgi.properties as mentioned in the description (r63396).
JVM options still need to be addressed.





[GLASSFISH-21016] Glassfish crashed unexplainable Created: 24/Mar/14  Updated: 07/Apr/14

Status: Open
Project: glassfish
Component/s: None
Affects Version/s: 4.0_b89_RC5
Fix Version/s: None

Type: Bug Priority: Major
Reporter: asmsys Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux v2201402190316899.yourvserver.net 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux



 Description   

My glassfish is running on at virtual machine based on ubuntu

root@v2201402190316899:/opt/oracle/glassfish4/glassfish/bin# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04.4 LTS
Release: 12.04
Codename: precise
root@v2201402190316899:/opt/oracle/glassfish4/glassfish/bin#

approximately once a week glassfish shuts down mysteriously and I have no explanation.

Logs until the crash are ok.

I have started my glassfish with start-domain --debug=true and hope for better logs helps me to analize the problem.

I would be glade if you could help me to get to the bottom of problem, because my glassfish runs important apps for me and I need that it runs always.



 Comments   
Comment by asmsys [ 06/Apr/14 ]

executing asadmin stop-domain results in a memory problem. here is the log. is there a possible relationsship with this issue?

root@v2201402190316899:/opt/oracle/glassfish4/glassfish/bin# cat hs_err_pid13375.log
#

  1. There is insufficient memory for the Java Runtime Environment to continue.
  2. Native memory allocation (malloc) failed to allocate 21757952 bytes for committing reserved memory.
  3. Possible reasons:
  4. The system is out of physical RAM or swap space
  5. In 32 bit mode, the process size limit was hit
  6. Possible solutions:
  7. Reduce memory load on the system
  8. Increase physical memory or swap space
  9. Check if swap backing store is full
  10. Use 64 bit Java on a 64 bit OS
  11. Decrease Java heap size (-Xmx/-Xms)
  12. Decrease number of Java threads
  13. Decrease Java thread stack sizes (-Xss)
  14. Set larger code cache with -XX:ReservedCodeCacheSize=
  15. This output file may be truncated or incomplete.
    #
  16. Out of Memory Error (os_linux.cpp:2761), pid=13375, tid=140563893012224
    #
  17. JRE version: (7.0_51) (build )
  18. Java VM: OpenJDK 64-Bit Server VM (24.45-b08 mixed mode linux-amd64 compressed oops)
  19. Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
    #

--------------- T H R E A D ---------------

Current thread (0x00007fd78c00a000): JavaThread "Unknown thread" [_thread_in_vm, id=13378, stack(0x00007fd794d94000,0x00007fd794e95000)]

Stack: [0x00007fd794d94000,0x00007fd794e95000], sp=0x00007fd794e931e0, free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x8f6303] VMError::report_and_die()+0x173
V [libjvm.so+0x464940] report_vm_out_of_memory(char const*, int, unsigned long, char const*)+0x70
V [libjvm.so+0x780491] os::Linux::commit_memory_impl(char*, unsigned long, unsigned long, bool)+0x1b1
V [libjvm.so+0x7804ec] os::pd_commit_memory(char*, unsigned long, unsigned long, bool)+0xc
V [libjvm.so+0x77a226] os::commit_memory(char*, unsigned long, unsigned long, bool)+0x26
V [libjvm.so+0x8f316f] VirtualSpace::initialize(ReservedSpace, unsigned long)+0x26f
V [libjvm.so+0x530217] CardGeneration::CardGeneration(ReservedSpace, unsigned long, int, GenRemSet*)+0x127
V [libjvm.so+0x40c22d] CompactingPermGenGen::CompactingPermGenGen(ReservedSpace, ReservedSpace, unsigned long, int, GenRemSet*, ContiguousSpace*, PermanentGenerationSpec*)+0xaf
V [libjvm.so+0x7c19bc] CompactingPermGen::CompactingPermGen(ReservedSpace, ReservedSpace, unsigned long, GenRemSet*, PermanentGenerationSpec*)+0xdc
V [libjvm.so+0x5312da] PermanentGenerationSpec::init(ReservedSpace, unsigned long, GenRemSet*)+0x1da
V [libjvm.so+0x522844] GenCollectedHeap::initialize()+0x434
V [libjvm.so+0x8c8b86] Universe::initialize_heap()+0xc6
V [libjvm.so+0x8c8ec4] universe_init()+0x64
V [libjvm.so+0x56734f] init_globals()+0x4f
V [libjvm.so+0x8af895] Threads::create_vm(JavaVMInitArgs*, bool*)+0x335
V [libjvm.so+0x5d0f5a] JNI_CreateJavaVM+0x5a
C [libjli.so+0x2a17] __strcat_chk+0x2a17

--------------- P R O C E S S ---------------

Java Threads: ( => current thread )

Other Threads:

=>0x00007fd78c00a000 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=13378, stack(0x00007fd794d94000,0x00007fd794e95000)]

VM state:not at safepoint (not fully initialized)

VM Mutex/Monitor currently owned by a thread: None

GC Heap History (0 events):
No events

Deoptimization events (0 events):
No events

Internal exceptions (0 events):
No events

Events (0 events):
No events

Dynamic libraries:
00400000-00401000 r-xp 00000000 fd:02 386348 /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
00600000-00601000 r--p 00000000 fd:02 386348 /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
00601000-00602000 rw-p 00001000 fd:02 386348 /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
025ca000-025eb000 rw-p 00000000 00:00 0 [heap]
d6800000-d6d30000 rw-p 00000000 00:00 0
d6d30000-e0e00000 rw-p 00000000 00:00 0
e0e00000-e1860000 rw-p 00000000 00:00 0
e1860000-f5a00000 rw-p 00000000 00:00 0
f6ec0000-100000000 rw-p 00000000 00:00 0
7fd789000000-7fd789270000 rwxp 00000000 00:00 0
7fd789270000-7fd78c025000 rw-p 00000000 00:00 0
7fd78c025000-7fd790000000 ---p 00000000 00:00 0
7fd79160e000-7fd791850000 rw-p 00000000 00:00 0
7fd791850000-7fd7918f1000 rw-p 00000000 00:00 0
7fd7918f1000-7fd7918f4000 rw-p 00000000 00:00 0
7fd7918f4000-7fd791944000 rw-p 00000000 00:00 0
7fd791944000-7fd79194a000 rw-p 00000000 00:00 0
7fd79194a000-7fd791a3d000 rw-p 00000000 00:00 0
7fd791a3d000-7fd791a3e000 rw-p 00000000 00:00 0
7fd791a3e000-7fd791a46000 r-xp 00000000 fd:02 386404 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libzip.so
7fd791a46000-7fd791c45000 ---p 00008000 fd:02 386404 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libzip.so
7fd791c45000-7fd791c46000 r--p 00007000 fd:02 386404 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libzip.so
7fd791c46000-7fd791c47000 rw-p 00008000 fd:02 386404 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libzip.so
7fd791c47000-7fd791c53000 r-xp 00000000 fd:02 154718 /lib/x86_64-linux-gnu/libnss_files-2.15.so
7fd791c53000-7fd791e52000 ---p 0000c000 fd:02 154718 /lib/x86_64-linux-gnu/libnss_files-2.15.so
7fd791e52000-7fd791e53000 r--p 0000b000 fd:02 154718 /lib/x86_64-linux-gnu/libnss_files-2.15.so
7fd791e53000-7fd791e54000 rw-p 0000c000 fd:02 154718 /lib/x86_64-linux-gnu/libnss_files-2.15.so
7fd791e54000-7fd791e5e000 r-xp 00000000 fd:02 154722 /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7fd791e5e000-7fd79205e000 ---p 0000a000 fd:02 154722 /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7fd79205e000-7fd79205f000 r--p 0000a000 fd:02 154722 /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7fd79205f000-7fd792060000 rw-p 0000b000 fd:02 154722 /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7fd792060000-7fd792077000 r-xp 00000000 fd:02 154734 /lib/x86_64-linux-gnu/libnsl-2.15.so
7fd792077000-7fd792276000 ---p 00017000 fd:02 154734 /lib/x86_64-linux-gnu/libnsl-2.15.so
7fd792276000-7fd792277000 r--p 00016000 fd:02 154734 /lib/x86_64-linux-gnu/libnsl-2.15.so
7fd792277000-7fd792278000 rw-p 00017000 fd:02 154734 /lib/x86_64-linux-gnu/libnsl-2.15.so
7fd792278000-7fd79227a000 rw-p 00000000 00:00 0
7fd79227a000-7fd792282000 r-xp 00000000 fd:02 154716 /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7fd792282000-7fd792481000 ---p 00008000 fd:02 154716 /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7fd792481000-7fd792482000 r--p 00007000 fd:02 154716 /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7fd792482000-7fd792483000 rw-p 00008000 fd:02 154716 /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7fd792483000-7fd7924ac000 r-xp 00000000 fd:02 386400 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libjava.so
7fd7924ac000-7fd7926ab000 ---p 00029000 fd:02 386400 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libjava.so
7fd7926ab000-7fd7926ac000 r--p 00028000 fd:02 386400 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libjava.so
7fd7926ac000-7fd7926ae000 rw-p 00029000 fd:02 386400 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libjava.so
7fd7926ae000-7fd7926bc000 r-xp 00000000 fd:02 386433 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libverify.so
7fd7926bc000-7fd7928bb000 ---p 0000e000 fd:02 386433 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libverify.so
7fd7928bb000-7fd7928bd000 r--p 0000d000 fd:02 386433 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libverify.so
7fd7928bd000-7fd7928be000 rw-p 0000f000 fd:02 386433 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/libverify.so
7fd7928be000-7fd7928c5000 r-xp 00000000 fd:02 154721 /lib/x86_64-linux-gnu/librt-2.15.so
7fd7928c5000-7fd792ac4000 ---p 00007000 fd:02 154721 /lib/x86_64-linux-gnu/librt-2.15.so
7fd792ac4000-7fd792ac5000 r--p 00006000 fd:02 154721 /lib/x86_64-linux-gnu/librt-2.15.so
7fd792ac5000-7fd792ac6000 rw-p 00007000 fd:02 154721 /lib/x86_64-linux-gnu/librt-2.15.so
7fd792ac6000-7fd792adb000 r-xp 00000000 fd:02 129043 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fd792adb000-7fd792cda000 ---p 00015000 fd:02 129043 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fd792cda000-7fd792cdb000 r--p 00014000 fd:02 129043 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fd792cdb000-7fd792cdc000 rw-p 00015000 fd:02 129043 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fd792cdc000-7fd792dd7000 r-xp 00000000 fd:02 154725 /lib/x86_64-linux-gnu/libm-2.15.so
7fd792dd7000-7fd792fd6000 ---p 000fb000 fd:02 154725 /lib/x86_64-linux-gnu/libm-2.15.so
7fd792fd6000-7fd792fd7000 r--p 000fa000 fd:02 154725 /lib/x86_64-linux-gnu/libm-2.15.so
7fd792fd7000-7fd792fd8000 rw-p 000fb000 fd:02 154725 /lib/x86_64-linux-gnu/libm-2.15.so
7fd792fd8000-7fd7930ba000 r-xp 00000000 fd:02 134377 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7fd7930ba000-7fd7932b9000 ---p 000e2000 fd:02 134377 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7fd7932b9000-7fd7932c1000 r--p 000e1000 fd:02 134377 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7fd7932c1000-7fd7932c3000 rw-p 000e9000 fd:02 134377 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7fd7932c3000-7fd7932d8000 rw-p 00000000 00:00 0
7fd7932d8000-7fd793d93000 r-xp 00000000 fd:02 386415 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
7fd793d93000-7fd793f93000 ---p 00abb000 fd:02 386415 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
7fd793f93000-7fd79402b000 r--p 00abb000 fd:02 386415 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
7fd79402b000-7fd79404e000 rw-p 00b53000 fd:02 386415 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
7fd79404e000-7fd79407b000 rw-p 00000000 00:00 0
7fd79407b000-7fd794091000 r-xp 00000000 fd:02 133378 /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7fd794091000-7fd794290000 ---p 00016000 fd:02 133378 /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7fd794290000-7fd794291000 r--p 00015000 fd:02 133378 /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7fd794291000-7fd794292000 rw-p 00016000 fd:02 133378 /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7fd794292000-7fd7942aa000 r-xp 00000000 fd:02 154723 /lib/x86_64-linux-gnu/libpthread-2.15.so
7fd7942aa000-7fd7944a9000 ---p 00018000 fd:02 154723 /lib/x86_64-linux-gnu/libpthread-2.15.so
7fd7944a9000-7fd7944aa000 r--p 00017000 fd:02 154723 /lib/x86_64-linux-gnu/libpthread-2.15.so
7fd7944aa000-7fd7944ab000 rw-p 00018000 fd:02 154723 /lib/x86_64-linux-gnu/libpthread-2.15.so
7fd7944ab000-7fd7944af000 rw-p 00000000 00:00 0
7fd7944af000-7fd7944b1000 r-xp 00000000 fd:02 154729 /lib/x86_64-linux-gnu/libdl-2.15.so
7fd7944b1000-7fd7946b1000 ---p 00002000 fd:02 154729 /lib/x86_64-linux-gnu/libdl-2.15.so
7fd7946b1000-7fd7946b2000 r--p 00002000 fd:02 154729 /lib/x86_64-linux-gnu/libdl-2.15.so
7fd7946b2000-7fd7946b3000 rw-p 00003000 fd:02 154729 /lib/x86_64-linux-gnu/libdl-2.15.so
7fd7946b3000-7fd794868000 r-xp 00000000 fd:02 154714 /lib/x86_64-linux-gnu/libc-2.15.so
7fd794868000-7fd794a68000 ---p 001b5000 fd:02 154714 /lib/x86_64-linux-gnu/libc-2.15.so
7fd794a68000-7fd794a6c000 r--p 001b5000 fd:02 154714 /lib/x86_64-linux-gnu/libc-2.15.so
7fd794a6c000-7fd794a6e000 rw-p 001b9000 fd:02 154714 /lib/x86_64-linux-gnu/libc-2.15.so
7fd794a6e000-7fd794a73000 rw-p 00000000 00:00 0
7fd794a73000-7fd794a80000 r-xp 00000000 fd:02 386426 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/jli/libjli.so
7fd794a80000-7fd794c7f000 ---p 0000d000 fd:02 386426 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/jli/libjli.so
7fd794c7f000-7fd794c80000 r--p 0000c000 fd:02 386426 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/jli/libjli.so
7fd794c80000-7fd794c81000 rw-p 0000d000 fd:02 386426 /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/jli/libjli.so
7fd794c81000-7fd794ca3000 r-xp 00000000 fd:02 154726 /lib/x86_64-linux-gnu/ld-2.15.so
7fd794ccc000-7fd794cd6000 rw-p 00000000 00:00 0
7fd794cd6000-7fd794d8c000 rw-p 00000000 00:00 0
7fd794d8c000-7fd794d94000 rw-s 00000000 fd:02 642769 /tmp/hsperfdata_root/13375
7fd794d94000-7fd794d97000 ---p 00000000 00:00 0
7fd794d97000-7fd794e9a000 rw-p 00000000 00:00 0
7fd794e9d000-7fd794ea0000 rw-p 00000000 00:00 0
7fd794ea0000-7fd794ea1000 r--p 00000000 00:00 0
7fd794ea1000-7fd794ea3000 rw-p 00000000 00:00 0
7fd794ea3000-7fd794ea4000 r--p 00022000 fd:02 154726 /lib/x86_64-linux-gnu/ld-2.15.so
7fd794ea4000-7fd794ea6000 rw-p 00023000 fd:02 154726 /lib/x86_64-linux-gnu/ld-2.15.so
7fff51bf3000-7fff51c14000 rw-p 00000000 00:00 0 [stack]
7fff51cc9000-7fff51cca000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

VM Arguments:
java_command: ./../lib/client/appserver-cli.jar stop-domain
Launcher Type: SUN_STANDARD

Environment Variables:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
SHELL=/bin/bash

Signal Handlers:
SIGSEGV: [libjvm.so+0x8f6d70], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGBUS: [libjvm.so+0x8f6d70], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGFPE: [libjvm.so+0x77aeb0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGPIPE: [libjvm.so+0x77aeb0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGXFSZ: [libjvm.so+0x77aeb0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGILL: [libjvm.so+0x77aeb0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: [libjvm.so+0x77b130], sa_mask[0]=0x00000000, sa_flags=0x10000004
SIGHUP: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGINT: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGTERM: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGQUIT: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000

--------------- S Y S T E M ---------------

OS:Ubuntu 12.04 (precise)
uname:Linux 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86_64
libc:glibc 2.15 NPTL 2.15
rlimit: STACK 8192k, CORE 0k, NPROC 7827, NOFILE 4096, AS infinity
load average:0.53 0.62 0.42

/proc/meminfo:
MemTotal: 1019536 kB
MemFree: 49816 kB
Buffers: 2212 kB
Cached: 20324 kB
SwapCached: 0 kB
Active: 890228 kB
Inactive: 18956 kB
Active(anon): 886716 kB
Inactive(anon): 240 kB
Active(file): 3512 kB
Inactive(file): 18716 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 68 kB
Writeback: 0 kB
AnonPages: 886708 kB
Mapped: 8416 kB
Shmem: 292 kB
Slab: 25536 kB
SReclaimable: 13112 kB
SUnreclaim: 12424 kB
KernelStack: 2960 kB
PageTables: 6188 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 509768 kB
Committed_AS: 1404320 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 14184 kB
VmallocChunk: 34359722000 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 49144 kB
DirectMap2M: 999424 kB
DirectMap1G: 0 kB

CPU:total 1 (1 cores per cpu, 1 threads per core) family 6 model 44 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, aes, tsc

/proc/cpuinfo:
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 44
model name : Intel(R) Xeon(R) CPU E5645 @ 2.40GHz
stepping : 2
microcode : 0x1
cpu MHz : 2393.998
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc up rep_good nopl pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes hypervisor lahf_lm
bogomips : 4787.99
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

Memory: 4k page, physical 1019536k(49816k free), swap 0k(0k free)

vm_info: OpenJDK 64-Bit Server VM (24.45-b08) for linux-amd64 JRE (1.7.0_51-b00), built on Jan 16 2014 18:37:59 by "buildd" with gcc 4.6.3

time: Sun Apr 6 22:28:14 2014
elapsed time: 0 seconds

root@v2201402190316899:/opt/oracle/glassfish4/glassfish/bin#





[GLASSFISH-21046] JMX Monitoring authentication fails when using LDAP Created: 21/Apr/14  Updated: 21/Apr/14

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 4.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: ecarpenter Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The JMX Remote monitoring does not work when Glassfish is configured to use LDAP for admin authentication. Changing the "Realm Name" of the Admin Service to a file based security realm does not help either.

In our environment we have a couple of clusters managed by the same DAS. The cluster's admin-realm is still file based(the default settting) and can one monitor the instances in the clusters without a problem. It is only the DAS that is failing with an authentication error.






[GLASSFISH-21296] the value of create in MDB is worng Created: 04/Feb/15  Updated: 18/Mar/15

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 4.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: lzg5039 Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

linux



 Description   

The method of MessageDrivenBeanStatsProvider'getCreateCount override the method of EjbMonitoringStatsProvider'getCreateCount,
but ManagedAttribute id of MessageDrivenBeanStatsProvider'getCreateCount is messagecount,
and return value of MessageDrivenBeanStatsProvider'getCreateCount is messageCount,
So I think the method name of MessageDrivenBeanStatsProvider is wrong.

org.glassfish.ejb.mdb.monitoring.stats.MessageDrivenBeanStatsProvider is as follows.

    @ManagedAttribute(id="messagecount")
    @Description( "Number of messages received for a message-driven bean")
    public CountStatistic getCreateCount() {
        return messageCount.getStatistic();
    }

com.sun.ejb.monitoring.stats.EjbMonitoringStatsProvider is as follows.

    @ManagedAttribute(id="createcount")
    @Description( "Number of times EJB create method is called")
    public CountStatistic getCreateCount() {
        return createStat.getStatistic();
    }


 Comments   
Comment by lzg5039 [ 04/Feb/15 ]

Modify the method name.The modified code is as follows.

The following source in the org.glassfish.ejb.mdb.monitoring.stats.MessageDrivenBeanStatsProvider

    @ManagedAttribute(id="messagecount")
    @Description( "Number of messages received for a message-driven bean")
    public CountStatistic getMessageCount() {
        return messageCount.getStatistic();
    }
Comment by zhouronghui [ 09/Feb/15 ]

Would you please give the operations and the error message?
By the way, the Title has a miss. "worng" should be "wrong".

Comment by lzg5039 [ 11/Mar/15 ]

Hi zhou

The following steps are excuted,the bug would happen
1.start imqbrokerd.
2.create jms resource.
3.write MDB APP(bussiness method print "Hello world" in server.log).
4.deploy MDB(receive jms message)
5.a client send a jms message,after MDB processed this message("Hello world" has been printed in server.log),the client send another jms message.

bug happens:
Because after MDB processed first message, the MDB receive second message, the MDB use the same instance to process first and second message.So the value of message-driven-bean-mon'createcount should be 1.But at this time use jconsole to see the value of message-driven-bean-mon'createcount which is 2.





[GLASSFISH-19811] usage of internal proprietary API in nucleus/common/common-util Created: 08/Mar/13  Updated: 08/Mar/13

Status: Open
Project: glassfish
Component/s: other
Affects Version/s: 4.0_b79
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Romain Grécourt Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks GLASSFISH-19812 Prevent usage of proprietary API" war... Open
Tags: build, common-util, maven, proprietary-api, warning

 Description   
[WARNING]  nucleus/common/common-util/src/main/java/com/sun/enterprise/universal/GFBase64Decoder.java:[50,45] BASE64Decoder is internal proprietary API and may be removed in a future release
[WARNING]  nucleus/common/common-util/src/main/java/com/sun/enterprise/universal/GFBase64Encoder.java:[50,46] BASE64Encoder is internal proprietary API and may be removed in a future release
[WARNING]  nucleus/common/common-util/src/main/java/com/sun/enterprise/universal/GFBase64Decoder.java:[50,45] BASE64Decoder is internal proprietary API and may be removed in a future release
[WARNING]  nucleus/common/common-util/src/main/java/com/sun/enterprise/universal/GFBase64Encoder.java:[50,46] BASE64Encoder is internal proprietary API and may be removed in a future release
[WARNING]  nucleus/common/common-util/src/main/java/com/sun/enterprise/universal/GFBase64Decoder.java:[50,45] BASE64Decoder is internal proprietary API and may be removed in a future release
[WARNING]  nucleus/common/common-util/src/main/java/com/sun/enterprise/universal/GFBase64Encoder.java:[50,46] BASE64Encoder is internal proprietary API and may be removed in a future release

Tests:

[WARNING]  nucleus/common/common-util/src/test/java/com/sun/enterprise/universal/BASE64DecoderTest.java:[82,16] BASE64Decoder is internal proprietary API and may be removed in a future release
[WARNING]  nucleus/common/common-util/src/test/java/com/sun/enterprise/universal/BASE64DecoderTest.java:[82,56] BASE64Decoder is internal proprietary API and may be removed in a future release
[WARNING]  nucleus/common/common-util/src/test/java/com/sun/enterprise/universal/BASE64DecoderTest.java:[83,16] BASE64Encoder is internal proprietary API and may be removed in a future release
[WARNING]  nucleus/common/common-util/src/test/java/com/sun/enterprise/universal/BASE64DecoderTest.java:[83,56] BASE64Encoder is internal proprietary API and may be removed in a future release


 Comments   
Comment by Romain Grécourt [ 08/Mar/13 ]

Tom, I was not able to find the right component.
Please forward this to the right component / assignee.

Thanks.

Comment by Tom Mueller [ 08/Mar/13 ]

Assigning to Byron since he was the last to modify this file.





[GLASSFISH-19768] Change restart-domain to use __locations (pid) instead of uptime Created: 04/Mar/13  Updated: 04/Mar/13

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 4.0_b78
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Tom Mueller Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The restart-domain command uses the "uptime" command to help figure out that the new server is running. It uses an algorithm that checks to see that the uptime of the new server is less than the uptime of the old server. However, if the server is restarted very quickly after it was started (so the old uptime is low) and the new server takes a long(er) time to start, then it is possible that this algorithm will think that the new server has never started.

A better method would be to use the __locations command which returns the pid. As soon as the pid is different and __locations returns, then we know that the new server is up and restart-domain can return.






[GLASSFISH-19649] Need Linux Startup script for Glassfish Created: 07/Feb/13  Updated: 07/Feb/13

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: tmpsa Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux


Status Whiteboard:

startup asadmin


 Description   

Problem: The system manager upgrades a production system using apt-get. One of the components is a DB system (say, MySql) which is then restarted. But since GF is not automagically shut down and re-started, it becomes terribly confused. JDBC connections are lost, and the JPA/EJB layers produce lots of exceptions for each client call. And afterwards, the webapp+glassfish remains in a confused state, making it inoperable, quickly filling all log files with exception tracebacks. A manual restart of GF is needed.

Possible solution: Distribute Glassfish with a 'official' endorsed and tested script for upstart. Then, if (say MySql is stopped by an upgrade, GF is automatically stopped first, and so on, in a nice sequence.

However, writing a 100% water-tight startup script for GF is non-trivial. It has to work excactly as asadmin start. For a discussion, see
http://java.net/projects/glassfish/lists/dev/archive/2012-02/message/9



 Comments   
Comment by Snjezana Sevo-Zenzerovic [ 07/Feb/13 ]

Moving to admin subcategory for evaluation.

Comment by Tom Mueller [ 07/Feb/13 ]

This looks like it might be related to service management, i.e., the create-service command.





[GLASSFISH-19500] Asadmin get --monitor returns remote failure when dynamic-reconfiguration-enabled=false Created: 08/Jan/13  Updated: 08/Jan/13

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: tesgoran Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Red Hat Linux 6.3



 Description   

I've several clusters with dynamic-reconfiguration-enabled=false.
When I use asadmin get --monitor to monitorize one instance I got "remote failure:" error.
But if I enable dynamic reconfiguration on a cluster I get the statistics for the resources of the instance.
I prefer not to enable dynamic reconfiguration on my clusters.
Is possible to monitorize the instances without dynamic-reconfiguration?

The steps to reproduce the problem:

  • Create "test" cluster with dynamic-reconfiguration-enabled=false
  • Create "test01" instance in "test" cluster.
  • asadmin get --monitor "test01.*" -> Remote failure: error
  • asadmin set test-config.dynamic-reconfiguration-enabled=true
  • asadmin get --monitor "test01.*" -> OK.


 Comments   
Comment by Tom Mueller [ 08/Jan/13 ]

Assigning to the monitoring component.





[GLASSFISH-20509] GlassFish incompatible with Debian 7 due to LSB incompliance of asadmin create-service Created: 11/May/13  Updated: 13/May/13

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: mkarg Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

GlassFish 3.1.2.2, Debian stable (7, "Wheezy"), OpenJDK 7u2+, amd64



 Description   

It seems GlassFish is incompatible with Debian 7 due to an LSB incompliance.

As a result GlassFish is neither starting at host boot, neither the update-rc.d script is able to modify the runlevel settings of the created Service. But manually invoking e. g. /etc/rc2.d/S20GlassFish_domain1 in fact does boot GlassFish correctly when executed manually at the command line!

An identified workaround is to execute the following commands:

  • sudo update-rc.d -f GlassFish_domain1 remove
  • sudo rm /etc/rcS.d/K20GlassFish_domain1
  • sudo update-rc.d GlassFish_domain1 Defaults

This sequence of commands actually fixed the problem very well, but certainly a future GlassFish release should produce rc*.d entries which do not need this Manual fix!

Notes:

  • "-f" is needed at update-rc.d because otherwise the command will not execute. The reason for this is that the rc*.d links produced by asadmin create-service seem to look "strange" in some way to the update-rc.d script. If I understand the output correctly, update-rc.d assumes that rc.local and GlassFish_domain1 scripts build a loop. The "-f" option Forces to ignore any such problems and simply kill all rc*.d links.
  • The manual deletion of the file in /etc/rcS.d is needed because even with the "-f" option, update-rc.d is not deleting the link in rcS.d. I suspect this is due to the fact that LSB defaults are K016 but not K016S?
  • I have not found out what actually the difference is between the original result of asadmin create-service, and the result of update-rc.d remove followed by update-rc.d default – besides the fact that the latter does not create a link in rcS.d, and sets sequence level 02 instead of 20.

I know that Debian is not officially supported, but as it looks as an LSB incomplicance to me, it would be good to fix this anyways.






[GLASSFISH-20498] Monitoring: misleading message or failure to register probe misses important data Created: 10/May/13  Updated: 21/Sep/15

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 4.0_b88_RC4
Fix Version/s: 4.1.1

Type: Bug Priority: Major
Reporter: marina vatkina Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

See GLASSFISH-19677 - the failed probe registration reports the probe class, not the actual failure:

WARNING: MNTG0201:Flashlight listener registration failed for listener class : com.sun.ejb.monitoring.stats.StatelessSessionBeanStatsProvider , will retry later

The actual cause is available only at FINE level:

FINE: Listener registration failed
java.lang.RuntimeException: Probe is not yet registered: glassfish:ejb:bean_null_MonTest_TestBean:methodEndEvent

But the actual probes are registered fine...



 Comments   
Comment by Terry2013 [ 31/Dec/14 ]

Hi Romain , Marina
I have reproduced the issue on the latest GlssfishV4.
I investigated a bit of source code , and found the logic in ejb-container is very odd.

com.sun.ejb.containers.BaseContainer
protected void createMonitoringRegistry() {
            //Please disregard the above code

            ejbProbeListener = getMonitoringStatsProvider(appName, modName, ejbName);
            ejbProbeListener.addMethods(getContainerId(), appName, modName, ejbName, getMonitoringMethodsArray());
            ejbProbeListener.register();★1

            if (_logger.isLoggable(Level.FINE)) {
	        _logger.log(Level.FINE, "Created MonitoringRegistry: " + 
                        EjbMonitoringUtils.getDetailedLoggingName(appName, modName, ejbName));
            }
	} catch (Exception ex) {
	    _logger.log(Level.SEVERE, COULD_NOT_CREATE_MONITORREGISTRYMEDIATOR, new Object[]{EjbMonitoringUtils.getDetailedLoggingName(appName, modName, ejbName), ex});
	}

        // Always create to avoid NPE
        try {
            ProbeProviderFactory probeFactory = ejbContainerUtilImpl.getProbeProviderFactory();
            String invokerId = EjbMonitoringUtils.getInvokerId(appName, modName, ejbName);
            ejbProbeNotifier = probeFactory.getProbeProvider(EjbMonitoringProbeProvider.class, invokerId);★2
            if (_logger.isLoggable(Level.FINE)) {
                _logger.log(Level.FINE, "Got ProbeProvider: " + ejbProbeNotifier.getClass().getName());
            }
        } catch (Exception ex) {
            ejbProbeNotifier = new EjbMonitoringProbeProvider();
            if (_logger.isLoggable(Level.FINE)) {
                _logger.log(Level.FINE, "Error getting the EjbMonitoringProbeProvider");
            }
        }

    }

In my understanding , Monitoring in GlassFish V4 need to register ProbeProvider and StatsProvider.
And StatProvider depend on ProbeProvider.So I think StatProvider's registration need after ProbeProvider's.
But in ejb container , StatProvider's registration (★1) is before ProbeProvider's(★2).
So the MNTG0201 message is output .

By the way,After I change ProbeProvider and ProbeProvider registration order, the MNTG0201 message is not output .

Comment by lzg5039 [ 20/Jan/15 ]

Hi Romain , Marina ,Terry2013

there is same probelm in the com.sun.ejb.containers.StatefulSessionContainer.registerMonitorableComponents
★2 order to register before ★1

 protected void registerMonitorableComponents() {
        super.registerMonitorableComponents();
        cacheProbeListener = new EjbCacheStatsProvider(sessionBeanCache,
                getContainerId(), containerInfo.appName, containerInfo.modName,
                containerInfo.ejbName);
        cacheProbeListener.register();★1

        try {
            ProbeProviderFactory probeFactory = ejbContainerUtilImpl.getProbeProviderFactory();
            String invokerId = EjbMonitoringUtils.getInvokerId(containerInfo.appName,
                    containerInfo.modName, containerInfo.ejbName);
            cacheProbeNotifier = probeFactory.getProbeProvider(EjbCacheProbeProvider.class, invokerId);★2
            if (_logger.isLoggable(Level.FINE)) {
                _logger.log(Level.FINE, "Got ProbeProvider: " + cacheProbeNotifier.getClass().getName());
            }
        } catch (Exception ex) {
            cacheProbeNotifier = new EjbCacheProbeProvider();
            if (_logger.isLoggable(Level.FINE)) {
                _logger.log(Level.FINE, "Error getting the EjbMonitoringProbeProvider");
            }
        }

        if (isHAEnabled) {
            sfsbStoreMonitor = new HAStatefulSessionStoreMonitor();
        } else {
            sfsbStoreMonitor = new StatefulSessionStoreMonitor();
        }
        sessionBeanCache.setStatefulSessionStoreMonitor(sfsbStoreMonitor);
        _logger.log(Level.FINE, "[SFSBContainer] registered monitorable");
    }

There is an another problem in the org.glassfish.persistence.ejb.entitybean.container.registerMonitorableComponents.
There is not EjbCacheProbeProvider in the registerMonitorableComponents(),but EjbCacheStatsProvider depends on the EjbCacheProbeProvider .
So I think it needs to register EjbCacheProbeProvider before ★1.

  protected void registerMonitorableComponents() {
        super.registerMonitorableComponents();
	if (readyStore != null) {
	    int confMaxCacheSize = cacheProp.maxCacheSize;
	    if (confMaxCacheSize <= 0) {
		confMaxCacheSize = Integer.MAX_VALUE;
	    }
	    this.cacheStatsProvider = new EntityCacheStatsProvider(
		    (BaseCache) readyStore, confMaxCacheSize);
	    //registryMediator.registerProvider(cacheStatsProvider);
            cacheProbeListener = new EjbCacheStatsProvider(cacheStatsProvider,
                    getContainerId(), containerInfo.appName, containerInfo.modName,
                    containerInfo.ejbName);
            cacheProbeListener.register();★1
	}
        poolProbeListener = new EjbPoolStatsProvider(entityCtxPool, 
                getContainerId(), containerInfo.appName, containerInfo.modName,
                containerInfo.ejbName);
        poolProbeListener.register();
        _logger.log(Level.FINE, "[Entity Container] registered monitorable");
    }





[GLASSFISH-20494] use the JavaVersion class from ASMainHelper instead of the JDK class Created: 08/May/13  Updated: 21/Sep/15

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 4.0_b87_RC3
Fix Version/s: 4.1.1

Type: Improvement Priority: Major
Reporter: Tom Mueller Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

A new class, JavaVersion has been introduced in the common-util module to handle Java versions. This class can be used within ASMainHelper to replace the usage of the JDK class. JDK.java can then be removed.

This enhancement is just for code cleanup.






[GLASSFISH-20535] Intermittent failure in admin-devtest-trunk-windows hudson job (start-domain/restart-domain/start-instance/start-local-instance) Created: 15/May/13  Updated: 21/Sep/15

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 4.0_b89_RC5
Fix Version/s: 4.1.1

Type: Bug Priority: Major
Reporter: Tom Mueller Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7



 Description   

The admin-devtest-trunk-windows hudson job is failing intermittently with failures in the following commands:

start-domain
restart-domain
start-instance (via start-cluster)
start-local-instance

The hudson job page is here. Recent failing jobs have been: #3000, #3002, #3003, #3005.
http://hudson-sca.us.oracle.com/view/GF%20Trunk/job/admin-devtests-trunk-windows/



 Comments   
Comment by Tom Mueller [ 15/May/13 ]

This problem has shown up in the 4.0 branch job too:

http://hudson-sca.us.oracle.com/job/admin-devtests-4.0-windows/2/

Comment by Byron Nevins [ 23/May/13 ]

This reminds me of the Heisenberg uncertainty principle. The moment I add diag code to the portion of the tests that fail - POOF. They don't fail anymore. And something seemingly completely different fails.

I've marked it for 4.0.1

I don't know what the real problem is. I'll just keep adding more and better diagnosis code directly into the tests and we'll find out in due course.

Comment by Byron Nevins [ 23/May/13 ]

The failed test on the 4.0 branch is restart-domain. This issue is about a start-cluster issue.

I suspect that it's gremlins that are only seen in automated tests. I can't ever reproduce it.

Comment by Tom Mueller [ 24/May/13 ]

Byron, John Wells suggested adding a build step to the end of the hudson job that would look for an ASMain processes that are left after the run and then run jstack on them. This might help in determining if the failure to start the server is caused by a deadlock.

Comment by marina vatkina [ 24/May/13 ]

I'm wondering if it is the same problem as I observed on the gf-transaction-cluster-devtest since November, until I changed the tests to dynamically determine the ports grabbed by the instances in a new cluster.





[GLASSFISH-20529] No easy way to specify LOW/HIGH monitoring level Created: 14/May/13  Updated: 21/Sep/15

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: 4.1.1

Type: Bug Priority: Major
Reporter: marina vatkina Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

There is no simple/declarative way to specify if the probe is available at LOW or HIGH monitoring level



 Comments   
Comment by Byron Nevins [ 14/May/13 ]

Research notes

> StatsProviderInfo has the info for a stats provider. It had the levels hacked into it later. So the constructor doesn't accept levels. The only way to set the level is to literally call the setter method at the exact right moment during registration.
>

That class is in GMBAL

hg clone https://hg.java.net/hg/gmbal~gf_common

Tip --> hunt around for StatsProviderInfo string and setConfigLevelk(sp?)

=====================

Comment by Byron Nevins [ 14/May/13 ]

More notes from an email thread:

Here it is:

public void registerStatsProvider(StatsProviderInfo spInfo) {
String configLevelStr = spInfo.getConfigLevel();

if (configLevelStr == null)

{ // Pick the highest in the configLevels spInfo.setConfigLevel(defaultConfigLevels[defaultConfigLevels.length-1]); }

==============

i.e. the StatsProviderInfo holds the config level (LOW or HIGH). Unfortunately that code is in GMBAL which I don't have right now.

If your code has something derived from StatsProviderInfo you can call setConfigLevel("LOW")





[GLASSFISH-20813] ConcurrentModificationException in GrizzlyService when booting GlassFish Created: 16/Sep/13  Updated: 17/Sep/13

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: jwells Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

On approximately 1 in 10 boots of GlassFish I get the following exception:

Shutting down server due to startup exception
java.util.ConcurrentModificationException
at java.util.Vector$Itr.checkForComodification(Vector.java:1156)
at java.util.Vector$Itr.next(Vector.java:1133)
at org.glassfish.external.probe.provider.StatsProviderManager.unregister(StatsProviderManager.java:100)
at com.sun.enterprise.v3.services.impl.monitor.GrizzlyMonitoring.registerThreadPoolStatsProvider(GrizzlyMonitoring.java:145)
at com.sun.enterprise.v3.services.impl.GlassfishNetworkListener.registerMonitoringStatsProviders(GlassfishNetworkListener.java:279)
at com.sun.enterprise.v3.services.impl.GlassfishNetworkListener.start(GlassfishNetworkListener.java:106)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy.start0(GrizzlyProxy.java:267)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy.start(GrizzlyProxy.java:241)
at com.sun.enterprise.v3.services.impl.GrizzlyService.createNetworkProxy(GrizzlyService.java:567)
at com.sun.enterprise.v3.services.impl.GrizzlyService.postConstruct(GrizzlyService.java:490)
at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:298)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:345)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:454)
at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:200)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2296)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:98)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:87)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:951)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:936)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)



 Comments   
Comment by oleksiys [ 17/Sep/13 ]

It's not related to Grizzly. Looks like it's monitoring (gmbal) related issue.





[GLASSFISH-20612] Meaningless Warnings In Log File Created: 06/Jun/13  Updated: 21/Sep/15

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 4.0_b89_RC5
Fix Version/s: 4.1.1

Type: Bug Priority: Major
Reporter: abien Assignee: Byron Nevins
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: fishcat

 Description   

Deployment of a Stateless Session Bean causes:

WARNING: Flashlight listener registration failed for listener class: com.sun.ejb.monitoring.stats.StatelessSessionBeanStatsProvider , will retry later

Deployment of a Singlet Bean causes:

WARNING: Flashlight listener registration failed for listener class: com.sun.ejb.monitoring.stats.SingletonBeanStatsProvider , will retry later

(Not tested with SFSB)

The same warnings already appeared on GF v3



 Comments   
Comment by Hong Zhang [ 06/Jun/13 ]

As these messages are from ejb monitoring probes, assign to ejb team for evaluation.

Comment by marina vatkina [ 06/Jun/13 ]

The warning comes from Flashlight code. EJB container can't do much about it.

Comment by ymajoros [ 20/Jun/13 ]

Looks like a duplicate of https://java.net/jira/browse/GLASSFISH-19677





[GLASSFISH-19254] WARNING Error during registration of FlashlightProbe java.lang.RuntimeException: JSR/RET are not supported with computeFrames option Created: 29/Oct/12  Updated: 29/Oct/12

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: tvlatas Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux host 2.6.18-308.0.0.0.1.el5xen #1 SMP Sat Feb 25 16:26:29 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
java version "1.7.0_07"
Java(TM) SE Runtime Environment (build 1.7.0_07-b10)
Java HotSpot(TM) Server VM (build 23.3-b01, mixed mode)



 Description   

I'm seeing a probe that is failing to be transformed by Flashlight:

[#|2012-10-29T07:34:24.441-0700|WARNING|44.0|javax.enterprise.system.tools.monitor|_ThreadID=1;_ThreadName=main;_TimeMillis=1351521264441;_LevelValue=900;_MessageID=NCLS-MON-0508;|Error during registration of FlashlightProbe
java.lang.RuntimeException: JSR/RET are not supported with computeFrames option
at org.objectweb.asm.Frame.execute(Frame.java:1159)
at org.objectweb.asm.MethodWriter.visitJumpInsn(MethodWriter.java:873)
at org.objectweb.asm.ClassReader.accept(ClassReader.java:1316)
at org.glassfish.flashlight.impl.client.ProbeProviderClassFileTransformer.transform(ProbeProviderClassFileTransformer.java:247)
at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:424)
at sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
at sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:144)
at org.glassfish.flashlight.impl.client.ProbeProviderClassFileTransformer.transform(ProbeProviderClassFileTransformer.java:180)
at org.glassfish.flashlight.impl.client.ProbeProviderClassFileTransformer.transformAll(ProbeProviderClassFileTransformer.java:121)
at org.glassfish.flashlight.impl.client.FlashlightProbeClientMediator.transformProbes(FlashlightProbeClientMediator.java:208)
at com.oracle.diagnostics.flightrecorder.impl.listener.JfrListener.evaluateProbeAndEventSettings(JfrListener.java:208)

Here is the probe it choked on:

Class: com.sun.mail.smtp.SMTPTransport
Probes: sendMessageStart::(Ljava/lang/String;)V
moduleprovidername: glassfish
modulename: javamail
probename: sendMessageStart

Looking at SMTPTransport, it is a V1.4 class with JSR/RET instructions being used there (ie: old style try/finally handling). The fix here would either be to have Flashlight not compute the frames for older version classes, or run the JSRInlinerAdapter across it to remove those usages (ie: so compute_frames won't choke on it). Note that JSRInlinerAdapter adds to the overhead, so probably want to conditionalize handling that for older classes only (ie: either way, conditionalize using compute_frames or conditionalize using the JSRInlinerAdapter)






[GLASSFISH-19439] create-service --serviceuser creates a service that fails to start Created: 14/Dec/12  Updated: 14/Dec/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Paul Davies Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

If the --serviceuser option of create-service is used to specify the user, a service is created that fails to start because of an ambiguous output re-direct error. The error occurs after a password is provided in response to a prompt from the start script.

A service was created to start the domain as user gfadmin by specifying --serviceuser gfadmin in the command to create the service:

$ sudo asadmin --user gfadmin --passwordfile gfadminpass create-service --serviceuser gfadmin domain1
The Service was created successfully. Here are the details:
Name of the service:domain1
Type of the service:Domain
Configuration location of the service:/etc/init.d/GlassFish_domain1
User account that will run the service: gfadmin
You have created the service but you need to start it yourself. Here are the most typical Linux commands of interest:

  • /etc/init.d/GlassFish_domain1 start
  • /etc/init.d/GlassFish_domain1 stop
  • /etc/init.d/GlassFish_domain1 restart

For your convenience this message has also been saved to this file: /export/glassfish3/glassfish/domains/domain1/PlatformServices.log
Command create-service executed successfully.

To test the service, the machine was rebooted, but the domain was not started. Starting the domain from the script results in a prompt for the user's password. After the password was provided, an Ambiguous output redirect error occurred:

$ /etc/init.d/GlassFish_domain1 start
Password:
Ambiguous output redirect.






[GLASSFISH-19416] the value of monitor becomes negative value Created: 07/Dec/12  Updated: 13/Dec/13

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: jifeng Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File JdbcConnPoolSatasProvider.patch     PNG File numConnFree.PNG    

 Description   

I execute under two commands:
1.set resources.jdbc-connection-pool.oracle_pool.pooling=false
2.enable-monitoring --modules jdbc-connection-pool=HIGH

then I get three jdbc connections from oracle_pool by my application.
at this time I find the value of "numconnfree-current" which become negative value



 Comments   
Comment by jifeng [ 07/Dec/12 ]

I think the code which is marked as "★" has problem
because there is no connection in the pool when the state of pool is unpooling

ConnectionPool.java

public String internalGetResource()
{   
    ........
    if (result != null) {
     if (maxConnectionUsage_ > 0) {
           result.incrementUsageCount();
     }
     if (poolLifeCycleListener != null) {
           poolLifeCycleListener.connectionUsed(result.getId());
          //Decrement numConnFree
          poolLifeCycleListener.decrementNumConnFree(); "★" 
     }
    }
}
Comment by jifeng [ 07/Dec/12 ]

I create a patch and it works fine, cloud you please confirm it.

Comment by jifeng [ 07/Dec/12 ]

I modify this method : JdbcConnPoolSatasProvider#decrementNumConnFreeEvent

Comment by jifeng [ 01/Aug/13 ]

Hi:

Byron Nevins

I find This phenomenon also occurs in glassfish v4 .

I added one line of code which is marked as "★" , and it works fine.

com.sun.enterprise.resource.pool.monitor.JdbcConnPoolStatsProvider.java
@ProbeListener(JDBC_PROBE_LISTENER + "decrementNumConnFreeEvent")
    public void decrementNumConnFreeEvent(
        @ProbeParam("poolName") String poolName,
        @ProbeParam("appName") String appName,
        @ProbeParam("moduleName") String moduleName
     ) {
     ......
        synchronized(numConnFree) {
            long numConnFreeToSet = (numConnFree.getCurrent() - 1 >= 0) ? numConnFree.getCurrent() - 1 : 0;★
            numConnFree.setCurrent(numConnFreeToSet);
        }
     }
    }

could you please confirm it and give me some suggestions?

Comment by boernd [ 13/Dec/13 ]

Hi,

we also got this phenomenon of negative values (gf 3.1.2.2) with the numConnUsed count. I changed the decrementConnectionUsedEvent method the same way:

            //Decrement numConnUsed counter
            synchronized(numConnUsed) {
                long numConnUsedToSet = (numConnUsed.getCurrent() - 1 >= 0) ? numConnUsed.getCurrent() - 1 : 0;                
                numConnUsed.setCurrent(numConnUsedToSet);
            }




[GLASSFISH-19404] "get --monitor server.transaction-service.activeids-current" does not work when tx monitor-level is set to LOW (both standalone instance and clustered instance) Created: 05/Dec/12  Updated: 05/Dec/12

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: 3.1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Wu Jie Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File parentTreeNode.png     Text File StatsProviderRegistry.patch    
Tags: jts

 Description   

when tx monitor-level is set to LOW by the commands below.
"enable-monitoring --modules transaction-service"
or
"set monitoring-service.module-monitoring-levels.transaction-service"

It could not get any inflight transaction info by the following command
"get --monitor server.transaction-service.activeids-current"



 Comments   
Comment by Wu Jie [ 05/Dec/12 ]

there is the same phenomenon on admin gui.

Comment by Wu Jie [ 05/Dec/12 ]

when execute "get --monitor server.transaction-service.activeids-current" ,
it will run into com.sun.enterprise.v3.admin.MonitoringReporter#runLocally() as listing in below.

    private void runLocally() {

        // don't run if this is DAS **and** DAS is not in the server list.
        // otherwise we are in an instance and definitely want to run!
        if (isDas() && !dasIsInList())
            return;

        // say the pattern is "something" -->
        // we want "server.something" for DAS and "i1.server.something" for i1
        // Yes -- this is difficult to get perfect!!!  What if user entered
        //"server.something"?

        String localPattern = prependServerDot(pattern);
        org.glassfish.flashlight.datatree.TreeNode tn = datareg.get(serverEnv.getInstanceName());

        if (tn == null) {
            // No monitoring data, so nothing to list
            // officially this is considered a "success"
            setSuccess(Strings.get("admin.get.monitoring.empty"));
            return;
        }

        List<org.glassfish.flashlight.datatree.TreeNode> ltn = tn.getNodes(localPattern);
        boolean singleStat = false;

        if (ltn == null || ltn.isEmpty()) {
            org.glassfish.flashlight.datatree.TreeNode parent = tn.getPossibleParentNode(localPattern);

            if (parent != null) {★
                ltn = new ArrayList<org.glassfish.flashlight.datatree.TreeNode>(1);
                ltn.add(parent);
                singleStat = true;
            }
        }

        if (!singleStat)
            localPattern = null; // signal to method call below.  localPattern was already used above...

        if (outputType == OutputType.GET)
            doGet(localPattern, ltn);
        else if (outputType == OutputType.LIST)
            doList(localPattern, ltn);

        if (plainReporter != null) {
            plainReporter.appendMessage(cliOutput.toString());
        }
    }

when tx monitor-level is HIGH, the parent TreeNode that is marked as "★" above is not null, which has value as the attached file "parentTreeNode.png".
but the parent TreeNode is null when tx monitor-level is LOW.

It seems that the transaction info is not registered into TreeNode(MonitoringRuntimeDataRegistry?) ...

Comment by marina vatkina [ 05/Dec/12 ]

"set monitoring-service.module-monitoring-levels.transaction-service" is not turning monitoring on, you should explicitly set it to LOW or HIGH

This is the code that turns monitoring on/off is:

           if (event.getSource() instanceof ModuleMonitoringLevels) {
                if (eventName.equals(ServerTags.TRANSACTION_SERVICE)) {
                    String newlevel = newValue.toString();
                    _logger.log(Level.FINE, "Changing transaction monitoring level");
                    if ("OFF".equals(newlevel)) {
                        tm.setMonitoringEnabled(false);
                    } else if ("LOW".equals(newlevel) || "HIGH".equals(newlevel)) {
                        tm.setMonitoringEnabled(true);
                    }
                } // else skip

So no value is considered as a no-op

Comment by Wu Jie [ 05/Dec/12 ]

I got the difference between "set monitoring-service.module-monitoring-levels.transaction-service" and "enable-monitoring --modules transaction-service".
but this issue is not related to the source above which is the segment of TransactionServiceConfigListener.

I observed that there is some difference between OFF->HIGH and OFF->LOW
when I change tx monitoring level from OFF to HIGH.
the runtime will run along the route as below.

Daemon Thread [pool-130-thread-1] (Suspended)
TransactionServiceStatsProvider.getActiveIds() line: 135
GeneratedMethodAccessor130.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 601
MethodInvokerImpl.getValue() line: 90
StatsProviderManagerDelegateImpl.resetChildNodeStatistics(String, List<String>, String) line: 556
StatsProviderManagerDelegateImpl.resetStatistics(StatsProviderRegistry$StatsProviderRegistryElement) line: 539
StatsProviderManagerDelegateImpl.enableStatsProvider(StatsProviderRegistry$StatsProviderRegistryElement) line: 389
StatsProviderManagerDelegateImpl.enableStatsProviders(String) line: 329
MonitoringBootstrap.handleLevelChange(String, String) line: 569
MonitoringBootstrap.changed(PropertyChangeEvent[]) line: 521
Transactions$ConfigListenerJob.process(ConfigListener) line: 401
Transactions$ConfigListenerJob.process(Object) line: 391
Transactions$ConfigListenerNotifier$1$1.call() line: 281
Transactions$ConfigListenerNotifier$1$1.call() line: 279
FutureTask$Sync.innerRun() line: 334
FutureTask<V>.run() line: 166
ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1110
ThreadPoolExecutor$Worker.run() line: 603
Thread.run() line: 722

but the situation which is OFF->LOW does not run the route above.

and look into related source, I doubt the source as following(org.glassfish.admin.monitor.StatsProviderManagerDelegateImpl).

    public void enableStatsProviders(String configElement) {
        //If monitoring-enabled is false, just return
        if (!getMonitoringEnabled())
            return;
        String configLevel = getMonitoringLevel(configElement);
        //Enable all the StatsProviders for a given configElement
        if (logger.isLoggable(Level.FINE))
            logger.fine("Enabling all the statsProviders for - " + configElement);
        List<StatsProviderRegistryElement> spreList = statsProviderRegistry.getStatsProviderRegistryElement(configElement);
        if (spreList == null)
            return;
        for (StatsProviderRegistryElement spre : spreList) {
            //Check to see if the enable is allowed
            // Not allowed if statsProvider is registered for Low and configLevel is HIGH
            boolean isEnableAllowed = spre.isEnableAllowed(configLevel);★
            if (!spre.isEnabled()) {
                //OFF->LOW, OFF->HIGH
                if (isEnableAllowed) {
                    enableStatsProvider(spre);
                }
            }
            else {
                //Disable if the stats were enabled, but current level is not allowed for these stats(HIGH->LOW) and
                // stats were registered at HIGH
                if (!isEnableAllowed) {
                    disableStatsProvider(spre);
                }// else, Dont do anything LOW->HIGH (stats were registered at LOW)
            }
        }
    }

the "isEnableAllowed" which is marked as "★" will be false in the situations as below.
・OFF->LOW
・HIGH->LOW

I think is there any wrong with the style.
and the "isEnableAllowed" should be true when OFF->LOW.

Comment by marina vatkina [ 05/Dec/12 ]

If this is the case, the error is in monitoring code.

Comment by Wu Jie [ 05/Dec/12 ]

the logic of "isEnableAllowed" is abstracted from org.glassfish.admin.monitor.StatsProviderRegistry as following:

        public boolean isEnableAllowed(String userConfigLevelStr) {
            Integer userConfigLevel = StatsProviderRegistry.configLevelsMap.get(userConfigLevelStr.toUpperCase());
            if ((userConfigLevel != null) && (userConfigLevel >= configLevel))//※the value of configLevel is always "1" which equals to "HIGH"
                return true;
            return false;
        }

※the value of configLevel is always "1" which equals to "HIGH" which related to the following source.

    static final String[] defaultConfigLevels = new String[] {"LOW","HIGH"};
    public static final Map<String, Integer> configLevelsMap = new ConcurrentHashMap();

    public StatsProviderRegistry(MonitoringRuntimeDataRegistry mrdr) {
        this.mrdr = mrdr;
        for (int i = 0; i < defaultConfigLevels.length; i++) {
            configLevelsMap.put(defaultConfigLevels[i].toUpperCase(), i);
        }
    }

    public void registerStatsProvider(StatsProviderInfo spInfo) {
        String configLevelStr = spInfo.getConfigLevel();

        if (configLevelStr == null) {
            // Pick the highest in the configLevels
            spInfo.setConfigLevel(defaultConfigLevels[defaultConfigLevels.length-1]);//★
        }

        StatsProviderRegistryElement spre =
                    new StatsProviderRegistryElement(spInfo);
        initialize(spre, spInfo.getConfigElement(), spInfo.getStatsProvider());
    }

the source which marked as "★" set the configLevelStr to "HIGH".

as the logic above, the "isEnableAllowed" returns false regardless of OFF->LOW or HIGH->LOW.
and I think the src logic may be modified as below.
① set configLevelStr to "LOW" as initial value.
② change the StatsProviderInfo#configLevel when monitoring level has been changed(LOW or HIGH).

Comment by Wu Jie [ 05/Dec/12 ]

I create a patch and it works fine. cloud you please confirm it.





[GLASSFISH-3598] Replace calls to System.setProperty with GFSystem.setProperty Created: 11/Sep/07  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 9.1peur1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: Byron Nevins Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 3,598
Tags: 3_1-exclude

 Description   

Replace every call to System.setPropert[y|ies]() with GFSystem.setPropert[y|ies]()

See http://blogs.sun.com/foo/entry/monitored_system_setproperty
for details on why.

I set the subcomponent to "Admin". It should be set to ALL subcomponents, but
Issue tracker won't allow that.



 Comments   
Comment by km [ 22/Sep/07 ]

BN.

Comment by km [ 22/Sep/07 ]

BN.

Comment by Tom Mueller [ 25/Jan/11 ]

Cleared the Fix version field since this issue isn't going to be fixed in 9.1peur1.

Comment by Byron Nevins [ 07/Feb/11 ]

Not difficult but messy – need to change dozens or hundreds of files...

The hard part is GFSystem implementation which is all done long ago.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-3589] jmx mbean deployment needs to be streamlined Created: 07/Sep/07  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 9.1peur1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: roshan_ail Assignee: Byron Nevins
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 3,589
Tags: 3_1-exclude

 Description   

The current process of deploying mbeans is cumbersome. At present you have to
put the jar under the lib folder in the glassfish install directory and then
call asadmin create-mbean for each mbean in the jar that you want to deploy.
This is a tedious process if there are a large number of mbeans and you have to
deploy them to a number of servers that are not clustered.

The deployment process should be simplified possibly by using a xml based
configuration file in which you define the mbean class and values for the
attributes.
The xml file should probably be independent of the jar file as this would
provide the flexibility of modifying the configuration at any point and have the
appserver load the new configuration.

The communication with kedar is in this thread.
http://forums.java.net/jive/thread.jspa?messageID=234484&#234484



 Comments   
Comment by Hong Zhang [ 10/Sep/07 ]

-> admin

Comment by km [ 10/Sep/07 ]

A brand new RFE that will almost certainly require "new" code!

Thanks for filing it. Assigning it to Byron.

Comment by Tom Mueller [ 25/Jan/11 ]

Cleared the Fix version field since this issue isn't going to be fixed in 9.1peur1.

Comment by Byron Nevins [ 07/Feb/11 ]

I don't know if this is still applicable. Leaving it alive until I get time to investigate

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-5979] Lack Of CallFlow Monitoring Created: 07/Sep/08  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: V3
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: abien Assignee: Byron Nevins
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: other
Platform: All


Issuezilla Id: 5,979

 Description   

Server: Glassfish v3 b23

In GF v2 CallFlow was a convenient way to easily debug and monitor the
application. GF v3 is lacking such "lightweight" functionality. At least a
call-tree should be visible in the console as well as "slowest" methods.

The monitoring features of GF v2 were unique comparing it to other, opensource
application servers...



 Comments   
Comment by Tom Mueller [ 06/Jan/11 ]

User is no longer on the project. Reassigning to default assignee for monitoring.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-5875] Watch out for http requests that are not accepted in time Created: 04/Sep/08  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: v2.1
Fix Version/s: future release

Type: Improvement Priority: Major
Reporter: kohsuke Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Linux


Issuezilla Id: 5,875
Tags: 3_1-exclude

 Description   

I'm not sure whether this should be under Grizzly or monitoring.

A trouble-shooting session with the internal customer of GlassFish (JavaFX)
revealed that one typical failure mode of GlassFish is the exhaustion of HTTP
request processing threads.

This can happen for a number of reasons, with a varying degree of consequences.
For example, if the # of request processing threads are too low (the system in
question had only 5 threads on a 4 core system), then it will cause GlassFish to
slow down.

Or if the web application has some problem and the request processing thread
never comes back (for example, it can be blocked during a processing on infinite
Object.wait() or dead lock), then GlassFish will eventually run out of all the
HTTP request threads, and GlassFish will look as if it hanged.

The problem here is that GlassFish doesn't provide any useful information to
point the system admin to the right direction, and it took someone from the
GlassFish team (namely, me) to look at it to figure out what's going on. This is
clearly undesirable.

So the RFEs here are two folds:

1. watch out for average delay an accepted socket connection is picked up by a
request handling thread. If it's longer than a certain threshold (say 1sec) and
if the # of HTTP request threads is too low compared to # of CPUs, then let the
admin know that there is a potential bottleneck.

2. if an accepted socket connection sits in the queue for too long, reject the
request and issue an error.

3. if no HTTP request thread picks up incoming connection for a certain period
of time (this could be long, like 1min), then it means all the threads are
hanging, so report an error with a stack trace indicating where the threads are
stuck.



 Comments   
Comment by kumara [ 01/Sep/09 ]

Changing version from 9.1.1 to v2.1 to reflect new name/version.

Comment by Tom Mueller [ 06/Jan/11 ]

User is no longer on the project. Reassigning to default assignee for monitoring.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-19371] asadmin start-domain times out after 600 seconds Created: 27/Nov/12  Updated: 17/Sep/13

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: beppino604 Assignee: Byron Nevins
Resolution: Unresolved Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Solaris 10, Java 1.6.0_26-b03


Tags: asadmin, timeout

 Description   

When trying to start a domain with a lot of applications to load the asadmin command times out after 600 seconds. However, the process continues to start and the service becomes available as expected. When listing the domains it says that the domain isn't running. This means that I can't stop the domain with asadmin anymore and the only option I have is to kill the OS process.

An additional headache is created by the fact that we want to manage GF via a Solaris SMF service. However, when the command returns a failure after 600 seconds, the SMF service goes to the state "maintenance" although the process continues to start. Not good in a Production environment.

Setting the environment variable AS_ADMIN_READTIMEOUT as suggested in another Jira defect doesn't make a difference. I tried setting it to 30 to force the process to fail after 30 seconds and I tried to set it to 3600000 but it didn't have an effect. Does it only work for the deploy command?

Long story short: the timeout should be configurable! Why would anyone make this a fixed value?

[app@xxxxxxxx] $ echo $AS_ADMIN_READTIMEOUT
360000
[app@xxxxxxxx] $ ./asadmin --passwordfile passwords start-domain
Waiting for xxxxxxxxxx to start ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
No response from the Domain Administration Server (xxxxxxxxxx) after 600 seconds.
The command is either taking too long to complete or the server has failed.
Please see the server log files for command status.
Please start with the --verbose option in order to see early messages.
Command start-domain failed.
[app@xxxxxxxx] $ ./asadmin --passwordfile passwords list-domains
xxxxxxxxxx not running
Command list-domains executed successfully.
[app@xxxxxxxx] $ ps -ef | grep java
app 4732 29327 0 06:55:40 pts/1 142:47 /usr/jdk/instances/jdk1.6.0/bin/sparcv9/java -cp /opt/be/glassfish-3.1.2.2/glas



 Comments   
Comment by Tom Mueller [ 27/Nov/12 ]

This issue is related to GLASSFISH-10076 which suggests some improvements in the wait start-domain determines the timeout.

At a minimum, the timeout should not be hardcoded as it currently is.
I'm leaving this a bug because of the behavior with list-domains. Even if start-domain times out, list-domains should still be able to detect the domain as running once the DAS is actually up.

Comment by beppino604 [ 28/Nov/12 ]

Thanks for accepting this one. By the way, this also happens with list-instances / start-instance.

Comment by markleadbitter [ 17/Sep/13 ]

Is this still an outstanding issue with it being raised in November last year? We are frequently hitting the same issue and would like to increase the default timeout.

Thanks.





[GLASSFISH-10106] Provide additional types for use with monitor command Created: 08/Oct/09  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: V3
Fix Version/s: future release

Type: Improvement Priority: Minor
Reporter: msreddy Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Sun


Issuezilla Id: 10,106

 Description   

Based on the use case that whenever a performance issue arises some one would
first go to top command (in unix) to detect the symptom for subsequent
diagnosis. Similarly, asadmin monitor command will be used to get the symptom.
This will be possible when we provide several 'types' to monitor. Presently, we
have very few types which are servlet, httplistener, jvm, webmodule.

We need to enhance this based on core monitoring infrastructure.



 Comments   
Comment by Tom Mueller [ 06/Jan/11 ]

User is no longer on the project. Reassigning to default assignee for monitoring.

Comment by Jennifer Chou [ 18/Jan/11 ]

Consider for 3.2 From v2.1.1 asadmin monitor --help there are the following types:

-type The type of statistics to moni
tor. Valid values are:

o connection

o connectionqueue

o connectorpool

o endpoint

o entitybean

o filecache

o httplistener

o httpservice

o jdbcpool

o jvm

o keepalive

o messagedriven

o servlet

o statefulsession

o statelesssession

o threadpool

o webmodule

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-8853] Multi-line StringStatistics corrupts all statistics Created: 23/Jul/09  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: V3
Fix Version/s: future release

Type: Improvement Priority: Minor
Reporter: marina vatkina Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Sun


Issue Links:
Dependency
blocks GLASSFISH-8955 headers and values are printed in the... Resolved
Issuezilla Id: 8,853

 Description   

This is the multiline output that is set into the current:

Transaction Id Status ElapsedTime(ms)
ComponentName ResourceNames

03000000AC5A66A9757068696C6C2C5431323438333832353731363336NoTransaction
3134 unknown
04000000AC5A66A9757068696C6C2C5431323438333832353731363336NoTransaction
2929 unknown

And this is what I get (see that even the CountStatistics are broken):

% asadmin get --monitor=true "server.transaction-service.*"
com.sun.enterprise.cli.framework.CommandException: remote failure: Unknown plain
text format. Response from the server: Signature-Version: 1.0
message: null
children: server.transaction-service.activecount-count = 0;server.tran
saction-service.activecount-description = Provides the number of tran
sactions that are currently active.;server.transaction-service.active
count-lastsampletime = 1248382567087;server.transaction-service.activ
ecount-name = ActiveCount;server.transaction-service.activecount-star
ttime = 1248382567087;server.transaction-service.activecount-unit = c
ount;server.transaction-service.activeids-description = Provides the
IDs of the transactions that are currently active a.k.a. in-flight tr
ansactions. Every such transaction can be rolled back after freezing
the transaction service.;server.transaction-service.activeids-lastsam
pletime = 1248382567089;server.transaction-service.activeids-name = A
ctiveIds;server.transaction-service.activeids-name-current =

Transa
ction Id Status ElapsedTim
e(ms) ComponentName ResourceNames

03000000AC5A6
6A9757068696C6C2C5431323438333832353731363336NoTransaction
3134 unknown
04000000AC5A66A97
57068696C6C2C5431323438333832353731363336NoTransaction 292
9 unknown ;server.transaction-se
rvice.activeids-starttime = 1248382567089;server.transaction-service.
activeids-unit = List;server.transaction-service.committedcount-count
= 0;server.transaction-service.committedcount-description = Provides
the number of transactions that have been committed.;server.transact
ion-service.committedcount-lastsampletime = 1248382567089;server.tran
saction-service.committedcount-name = CommittedCount;server.transacti
on-service.committedcount-starttime = 1248382567089;server.transactio
n-service.committedcount-unit = count;server.transaction-service.roll
edbackcount-count = 0;server.transaction-service.rolledbackcount-desc
ription = Provides the number of transactions that have been rolled b
ack.;server.transaction-service.rolledbackcount-lastsampletime = 1248
382567089;server.transaction-service.rolledbackcount-name = Rolledbac
kCount;server.transaction-service.rolledbackcount-starttime = 1248382
567089;server.transaction-service.rolledbackcount-unit = count;server
.transaction-service.state-description = Indicates if the transaction
service has been frozen;server.transaction-service.state-lastsamplet
ime = 1248382567091;server.transaction-service.state-name = State;ser
ver.transaction-service.state-name-current = False;server.transaction
-service.state-starttime = 1248382567091;server.transaction-service.s
tate-unit = String
use-main-children-attribute: true
children-type: null
exit-code: SUCCESS

Name: server.transaction-service.activecount-starttime = 1248382567087
message: server.transaction-service.activecount-starttime = 1248382567
087

Name: server.transaction-service.committedcount-starttime = 1248382567
089
message: server.transaction-service.committedcount-starttime = 1248382
567089

Name: server.transaction-service.committedcount-unit = count
message: server.transaction-service.committedcount-unit = count

Name: server.transaction-service.rolledbackcount-name = RolledbackCoun
t
message: server.transaction-service.rolledbackcount-name = RolledbackC
ount

Name: server.transaction-service.state-lastsampletime = 1248382567091
message: server.transaction-service.state-lastsampletime = 12483825670
91

Name: server.transaction-service.rolledbackcount-description = Provide
s the number of transactions that have been rolled back.
message: server.transaction-service.rolledbackcount-description = Prov
ides the number of transactions that have been rolled back.

Name: server.transaction-service.rolledbackcount-starttime = 124838256
7089
message: server.transaction-service.rolledbackcount-starttime = 124838
2567089

Name: server.transaction-service.activecount-lastsampletime = 12483825
67087
message: server.transaction-service.activecount-lastsampletime = 12483
82567087

Name: server.transaction-service.rolledbackcount-unit = count
message: server.transaction-service.rolledbackcount-unit = count

Name: server.transaction-service.state-name = State
message: server.transaction-service.state-name = State

Name: server.transaction-service.rolledbackcount-lastsampletime = 1248
382567089
message: server.transaction-service.rolledbackcount-lastsampletime = 1
248382567089

Name: server.transaction-service.committedcount-description = Provides
the number of transactions that have been committed.
message: server.transaction-service.committedcount-description = Provi
des the number of transactions that have been committed.

Name: server.transaction-service.activecount-name = ActiveCount
message: server.transaction-service.activecount-name = ActiveCount

Name: server.transaction-service.activeids-unit = List
message: server.transaction-service.activeids-unit = List

Name: server.transaction-service.activeids-description = Provides the
IDs of the transactions that are currently active a.k.a. in-flight tr
ansactions. Every such transaction can be rolled back after freezing
the transaction service.
message: server.transaction-service.activeids-description = Provides t
he IDs of the transactions that are currently active a.k.a. in-flight
transactions. Every such transaction can be rolled back after freezi
ng the transaction service.

Name: server.transaction-service.rolledbackcount-count = 0
message: server.transaction-service.rolledbackcount-count = 0

Name: server.transaction-service.state-name-current = False
message: server.transaction-service.state-name-current = False

Name: server.transaction-service.state-unit = String
message: server.transaction-service.state-unit = String

Name: server.transaction-service.activecount-unit = count
message: server.transaction-service.activecount-unit = count

Name: server.transaction-service.activecount-count = 0
message: server.transaction-service.activecount-count = 0

Name: server.transaction-service.committedcount-count = 0
message: server.transaction-service.committedcount-count = 0

Name: server.transaction-service.committedcount-lastsampletime = 12483
82567089
message: server.transaction-service.committedcount-lastsampletime = 12
48382567089

Name: server.transaction-service.activeids-name-current =

Transactio
n Id Status ElapsedTime(ms
) ComponentName ResourceNames

03000000AC5A66A97
57068696C6C2C5431323438333832353731363336NoTransaction 313
4 unknown
04000000AC5A66A975706
8696C6C2C5431323438333832353731363336NoTransaction 2929
unknown
message: server.transaction-service.activeids-name-current =

Transac
tion Id Status ElapsedTime
(ms) ComponentName ResourceNames

03000000AC5A66
A9757068696C6C2C5431323438333832353731363336NoTransaction
3134 unknown
04000000AC5A66A975
7068696C6C2C5431323438333832353731363336NoTransaction 2929
unknown

Name: server.transaction-service.state-description = Indicates if the
transaction service has been frozen
message: server.transaction-service.state-description = Indicates if t
he transaction service has been frozen

Name: server.transaction-service.activeids-starttime = 1248382567089
message: server.transaction-service.activeids-starttime = 124838256708
9

Name: server.transaction-service.activecount-description = Provides th
e number of transactions that are currently active.
message: server.transaction-service.activecount-description = Provides
the number of transactions that are currently active.

Name: server.transaction-service.committedcount-name = CommittedCount
message: server.transaction-service.committedcount-name = CommittedCou
nt

Name: server.transaction-service.state-starttime = 1248382567091
message: server.transaction-service.state-starttime = 1248382567091

Name: server.transaction-service.activeids-name = ActiveIds
message: server.transaction-service.activeids-name = ActiveIds

Name: server.transaction-service.activeids-lastsampletime = 1248382567
089
message: server.transaction-service.activeids-lastsampletime = 1248382
567089

Command get failed.



 Comments   
Comment by Nazrul [ 09/Sep/09 ]

Prashanth -> Nachiappan

Comment by abbagani [ 25/Sep/09 ]

Formatting the output in the backend is wrong design. We were doing this wrong
in V2 and there is no reason to continue doing this for v3. We should return
proper objects to the client and let the client design what format he/she wants
it to be displayed. More important as we have varying degrees of clients here
(GUI,asadmin,DTrace,Scripting-client and MBeans).

The proper way to handle this is enclose your transaction information into a
Statistic object and then return that array back to the caller.
You will first need to define your Transaction Statistic object which will
contain (Transaction Id, Status, ElapsedTime(ms), ComponentName, ResourceNames)
which will extend StatisticImpl. Please see
org.glassfish.webservices.monitoring.DeployedEndpointData as an example.
If you have multiples of this Objects to be returned, you would define your
Transactions Stats object which will implement Stats interface, and return your
array from here. See the inner class MyStats in
org.glassfish.webservices.monitoring.WebServiceStatsProvider on how they did
this. Once you have this, all the clients will handle the output in their own
way and you will not have to worry about the formatting for them.

Marking the bug as P3 and transferring to Transaction owner

Comment by marina vatkina [ 26/Sep/09 ]

There is a separate bug in transactions output (see the issue that depends on
this one).

1. It's still a bug in the monitoring that a multi-line String (whether right or
wrong) not only can't be displayed properly, but it corrupts output for other
monitoring data.

2. If there are already 3 places that need a multi-line output, and the existing
code can't handle a String that represents more than 1 line, either the
StringStatisticImpl should be able to take an Array or a List of Strings or a
separate MultilineStringStatisticImpl should be added.

Comment by abbagani [ 27/Sep/09 ]

What other 3 places are you referring to, can you please state that. No other
modules has this requirement.

I see the same example being referred in the 8955 bugs as well.
As far as this example is concerned, you shouldn't be formatting the output on
your side. They should be returned as statistic objects to the clients. Please
plan on fixing it.

Comment by abbagani [ 28/Sep/09 ]

Trasferring to Marina.

Comment by marina vatkina [ 30/Sep/09 ]

As discussed on the eng meeting...

Comment by Nazrul [ 30/Sep/09 ]

Getting help from Sreeni

Comment by msreddy [ 01/Oct/09 ]

Thanks to Nachi for help on this bug.
Suggested to Marina to use "%%%EOL%%%" instead of "/n" as the data sent back to
client is done using Zip Manifest. I have tested it on Mac, Linux and Solaris
X86 and it works fine with the above change.

The code diff for Marina is:

jyothi[77]: svn diff
src/main/java/com/sun/enterprise/transaction/monitoring/TransactionServiceStatsProvider.java
Index:
src/main/java/com/sun/enterprise/transaction/monitoring/TransactionServiceStatsProvider.java
===================================================================

src/main/java/com/sun/enterprise/transaction/monitoring/TransactionServiceStatsProvider.java
(revision 32153)
+++
src/main/java/com/sun/enterprise/transaction/monitoring/TransactionServiceStatsProvider.java
(working copy)
@@ -142,11 +142,12 @@
if (aList.size() > 0)

{ // XXX strBuf.append("\n\n"); + strBuf.append("%%%EOL%%%"); appendColumn(strBuf, "Transaction Id", COLUMN_LENGTH+15); appendColumn(strBuf, "Status", COLUMN_LENGTH); appendColumn(strBuf, "ElapsedTime(ms)", COLUMN_LENGTH); appendColumn(strBuf, "ComponentName", componentNameLength); - strBuf.append("ResourceNames "); // XXX \n"); + strBuf.append("ResourceNames %%%EOL%%%"); }

for (int i=0; i < aList.size(); i++) {
@@ -154,6 +155,7 @@
String txnId = txnBean.getId();

// XXX strBuf.append("\n");
+ strBuf.append("%%%EOL%%%");
_logger.fine("=== Processing txnId: " + txnId);
appendColumn(strBuf, txnId, COLUMN_LENGTH+15);
appendColumn(strBuf, txnBean.getStatus(), COLUMN_LENGTH);
jyothi[78]:

The corresponding output of get is:

jyothi[65]: ./asadmin get -m "server.transaction-service.*"
server.transaction-service.activecount-count = 1
server.transaction-service.activecount-description = Provides the number of
transactions that are currently active.
server.transaction-service.activecount-lastsampletime = 1254449158267
server.transaction-service.activecount-name = ActiveCount
server.transaction-service.activecount-starttime = 1254449062341
server.transaction-service.activecount-unit = count
server.transaction-service.activeids-current =
Transaction Id Status ElapsedTime(ms)
ComponentName ResourceNames

0000000000000001_00 Active 20930
org.nachi.transactiontest.ejb.SimpleSessionBean jdbc/__default,
server.transaction-service.activeids-description = Provides the IDs of the
transactions that are currently active a.k.a. in-flight transactions. Every such
transaction can be rolled back after freezing the transaction service.
server.transaction-service.activeids-lastsampletime = 1254449179193
server.transaction-service.activeids-name = ActiveIds
server.transaction-service.activeids-starttime = 1254449062341
server.transaction-service.activeids-unit = List
server.transaction-service.committedcount-count = 0
server.transaction-service.committedcount-description = Provides the number of
transactions that have been committed.
server.transaction-service.committedcount-lastsampletime = -1
server.transaction-service.committedcount-name = CommittedCount
server.transaction-service.committedcount-starttime = 1254449062341
server.transaction-service.committedcount-unit = count
server.transaction-service.dotted-name = server.transaction-service
server.transaction-service.rolledbackcount-count = 0
server.transaction-service.rolledbackcount-description = Provides the number of
transactions that have been rolled back.
server.transaction-service.rolledbackcount-lastsampletime = -1
server.transaction-service.rolledbackcount-name = RolledbackCount
server.transaction-service.rolledbackcount-starttime = 1254449062341
server.transaction-service.rolledbackcount-unit = count
server.transaction-service.state-current = True
server.transaction-service.state-description = Indicates if the transaction
service has been frozen
server.transaction-service.state-lastsampletime = 1254449179194
server.transaction-service.state-name = State
server.transaction-service.state-starttime = 1254449062341
server.transaction-service.state-unit = String

Command get executed successfully.
jyothi[66]:

Based on discussion with Marina, lowering the priority to P4 to fix the
infrastructure as enhancement.

Comment by Tom Mueller [ 06/Jan/11 ]

Assigned user is no longer on project. Reassigning to category owner.

Comment by Byron Nevins [ 07/Feb/11 ]

Unfortunately we are forced to use the Manifest object for returning data to asadmin.

One of the annoying problems with the Manifest object is that it will happily accept embedded linefeeds when you WRITE it. And then explode later when it is READ!

The solution is to always encode linefeeds that are part of your formatting for a String like so –

import com.sun.enterprise.universal.collections.ManifestUtils;
myString = ManifestUtils.encode(myString);

Comment by marina vatkina [ 23/Mar/11 ]

Byron, either close this bug as won't fix, or reject linefeeds in the monitoring strings. I can't do anything with the infrastructure, and the jts code already replaces linefeeds with the special marker

Comment by Byron Nevins [ 23/Mar/11 ]

Marina,

Your comment does not compute. Please clarify.

I'll mark it for 3.2 for now.

I may just run all monitoring strings through the ONE extra line of code to guarantee the manifest object won't explode in asadmin
Clearly monitoring should be more defensive. Monitoring callers are ignoring the rules of the road!

Comment by marina vatkina [ 23/Mar/11 ]

My comment was about reassigning this bug to you - it was assigned to me, though I'm not working on monitoring infrastructure. HTH.

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-14844] MiniXmlParserTest takes upto a minute outside SWAN Created: 28/Nov/10  Updated: 30/Nov/10

Status: Open
Project: glassfish
Component/s: other
Affects Version/s: 3.1
Fix Version/s: future release

Type: Bug Priority: Minor
Reporter: Sanjeeb Sahoo Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux Sahoo 2.6.32-22-generic #33-Ubuntu SMP Wed Apr 28 13:27:30 UTC 2010 i686 GNU/Linux
java version "1.6.0_21"
Java(TM) SE Runtime Environment (build 1.6.0_21-b06)
Java HotSpot(TM) Server VM (build 17.0-b16, mixed mode)



 Description   

This is with latest workspace while doing a build outside SWAN:

Running com.sun.enterprise.universal.xml.MiniXmlParserTest
28 Nov, 2010 10:29:45 PM com.sun.enterprise.universal.xml.MiniXmlParser findDomainNameAndEnd
INFO: "Warning: No domain name property found. I was looking for a property element under the domain element that looks like this: <property value="domain1" name="administrative.domain.name" />
Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.5 sec

Even time command (time mvn -o surefire:test -Dtest=MiniXmlParserTest) reports similar output. With or without env variable AS_NO_REVERSE_DNS=TRUE, the aforementioned command now takes anywhere between 35 to 50 seconds on my system. So, I do believe there is some network layer caching going on which has brought the total time down from what I first observed last night. Yes, I am online, but outside SWAN. I think if just one unit test takes 1 minute where as the entire v3 build takes 15 minutes is a waste of developer time. So excluding this test until we can bring it down.



 Comments   
Comment by Byron Nevins [ 29/Nov/10 ]

I can't fix what I can't reproduce. I just did exactly what you did.

1) run the test normally (2 seconds)
2) run the test after UNPLUGGING my ethernet cable (2 seconds)

There were no problems of any kind.

The problem seems to exist only in your environment.

Comment by Byron Nevins [ 29/Nov/10 ]

Lowered priority since:

(1) it is a build-issue.
(2) it is known to occur only for one developer

Comment by Sanjeeb Sahoo [ 29/Nov/10 ]

As long as you don't enable the test, it does not really matter what the priority of this bug is. I created this bug so that we can capture discussion around this bug in one place if we have any. In other words, please don't enable the test in default build profile.





[GLASSFISH-6422] [UB] [Release Note] newly created vs didn't show under HTTPService monitoring Created: 03/Oct/08  Updated: 02/Dec/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: V3
Fix Version/s: 4.0

Type: Bug Priority: Minor
Reporter: yifeng1 Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Solaris
Platform: PC


Attachments: JPEG File monitoring1.JPG    
Issuezilla Id: 6,422
Status Whiteboard:

gfv3-prelude-excluded


 Description   

b27, open solaris, IE 7

after creation of a new virtual server, I go to monotoring tab, from the drop
down list, I didn't see this new virtual server. I have tried reload browser,
click refresh page, still doesn't show. CLI command doesn't have new virtual
server listed either.

to reproduce:

1. turn monitoring level to HIGH and save
2. configuration->virtual servers>click new>input myvs1 as id->click save
3. configuration->http listeners>click new->
put name as myhs1, network address 0.0.0.0 listener port 6666 default virtual
server myvs1-> click next--> click finish
4. access http://myhost:6666 , make sure the new http listener for myvs1 is working
5. application server->monitor->HTTP Service
notice the virtual server named myvs1 is not on the pulldown list

6. corresponding CLI command doesn't return the stats for myvs1 either

  1. uname -a/asadmin get --monitor=true "server.http-service.*"
    server.http-service.__asadmin.request.count5xx-count = 0
    server.http-service.server.request.count404-count = 0
    server.http-service.__asadmin.request.count503-count = 0
    server.http-service.__asadmin.request.requestCount = 0
    server.http-service.__asadmin.request.processingTime = NaN
    server.http-service.server.request.count200-count = 0
    server.http-service.server.request.count2xx-count = 0
    server.http-service.server.request.count302-count = 0
    server.http-service.server.request.countother-count = 0
    server.http-service.server.request.requestCount = 0
    server.http-service.server.request.count4xx-count = 0
    server.http-service.server.request.count400-count = 0
    server.http-service.__asadmin.request.count401-count = 0
    server.http-service.__asadmin.request.countother-count = 0
    server.http-service.__asadmin.request.count400-count = 0
    server.http-service.server.request.errorCount = 0
    server.http-service.server.request.maxTime = 0
    server.http-service.__asadmin.request.count403-count = 0
    server.http-service.__asadmin.request.count302-count = 0
    server.http-service.server.request.count5xx-count = 0
    server.http-service.__asadmin.request.count3xx-count = 0
    server.http-service.__asadmin.request.count2xx-count = 0
    server.http-service.__asadmin.request.count4xx-count = 0
    server.http-service.__asadmin.request.errorCount = 0
    server.http-service.server.request.count503-count = 0
    server.http-service.__asadmin.request.count200-count = 0
    server.http-service.server.request.count3xx-count = 0
    server.http-service.server.request.count401-count = 0
    server.http-service.server.request.processingTime = NaN
    server.http-service.__asadmin.request.count304-count = 0
    server.http-service.__asadmin.request.count404-count = 0
    server.http-service.server.request.count403-count = 0
    server.http-service.server.request.count304-count = 0
    server.http-service.__asadmin.request.maxTime = 0


 Comments   
Comment by yifeng1 [ 03/Oct/08 ]

Created an attachment (id=1929)
add attachment

Comment by yifeng1 [ 03/Oct/08 ]

add one attachment, in the picture, left side window shows the newly created
http listener myhs1 is running on myvs1, right side window is Admin console, the
drop down list doesn't have myvs1.

Comment by Anissa Lam [ 03/Oct/08 ]

This is a monitoring issue, related to dynamic reconfiguration.
The newly created VS will show up after server restart.
Transferring to monitoring.

Comment by Nazrul [ 08/Oct/08 ]

Discussed this with Jerome and Abhijit. Will be fixed after GFv3 Prelude. User
will need to restart to see stats for newly created virtual server.

We may release note this.

Comment by Nazrul [ 15/Oct/08 ]

Need to release note for GFv3 Prelude.

Comment by kumara [ 24/Oct/08 ]

Prashanth: Please be sure to get a release note entry added for this issue. Excluding from prelude release.

Comment by kumara [ 24/Oct/08 ]

Reclassifying as P4 because these issues are not must fix for prelude release.
This issue will be scrubbed after prelude release and will be given the right
priority for v3 final release.

Comment by Gail Risdal [ 08/Dec/09 ]

Added to v3 Release Notes.

Comment by Tom Mueller [ 06/Jan/11 ]

User is no longer on the project. Reassigning to default assignee for monitoring.

Comment by Jennifer Chou [ 18/Jan/11 ]

Consider for 3.2.





[GLASSFISH-6263] Add --filter option to monitor command Created: 23/Sep/08  Updated: 05/Apr/11

Status: Open
Project: glassfish
Component/s: monitoring
Affects Version/s: V3
Fix Version/s: future release

Type: New Feature Priority: Minor
Reporter: sankarpn Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 6,263
Status Whiteboard:

gfv3-prelude-excluded


 Description   

C:\work\admincli\ws\v3\glassfishv3-prelude\glassfish>asadmin list-applications

webmodmonitoring <web>
jdbcapp1webmod1 <web>

Command list-applications executed successfully.

There are 2 webmodules deployed in the server.

Executing this command should fail since there is a ambiguity in deciding for
which module it is supposed to return the stats. But the command doesn't fail
and deisplays the stats.

C:\work\admincli\ws\v3\glassfishv3-prelude\glassfish>asadmin monitor --type=webm
odule --interval=10 server
asc ast rst st ajlc mjlc tjlc aslc mslc tslc
0 0 0 0 0 0 0 16 16 16

0 0 0 0 0 0 0 16 16 16



 Comments   
Comment by kumara [ 23/Sep/08 ]

v3 defect tracking

Comment by janey [ 23/Sep/08 ]

reassign to Sreeni.

Comment by msreddy [ 24/Sep/08 ]

In prelude, we are not supporting filters for web module which is contemplated
for post prelude.

Comment by kumara [ 24/Oct/08 ]

Reclassifying as P4 because these issues are not must fix for prelude release.
This issue will be scrubbed after prelude release and will be given the right
priority for v3 final release.

Comment by Tom Mueller [ 23/Jun/10 ]

Sreenivas no longer on project.

Comment by Jennifer Chou [ 20/Jan/11 ]

Considering adding support for --filter in monitor command for 3.2

Comment by Tom Mueller [ 05/Apr/11 ]

A fix for this issue was initially identified for possible inclusion in the 3.2 release, but after further 3.2 planning, the feature or improvement did not make the cut. This issue is being targeted for a future release. If based on a reevaluation, it is targeted for 3.2, then update the "fix version" again.





[GLASSFISH-18182] Error message too long, hard to read in Admin Console Created: 13/Jan/12  Updated: 13/Jan/12

Status: Open
Project: glassfish
Component/s: distributed management
Affects Version/s: 3.1.2_b17
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: lidiam Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

build ogs-3.1.2-b17.zip


Attachments: JPEG File node-create-error.JPG    
Tags: 312_qa

 Description   

Currently when user tries to install a node on a remote host to a directory where glassfish is already installed the following is printed in Admin Console:

An error has occurred
Successfully connected to j2eetest@tuppy using keyfile /export/home/j2eetest/.ssh/id_rsa Command install-node-ssh failed. Ignoring unrecognized element schedules at Line number = 57 Column number = 18 System Id = file:/export/home/j2eetest/3.1.2/glassfish3/glassfish/domains/domain1/config/domain.xml Public Id = null Location Uri= file:/export/home/j2eetest/3.1.2/glassfish3/glassfish/domains/domain1/config/domain.xml CharacterOffset = 3425 Ignoring unrecognized element backup-configs at Line number = 62 Column number = 23 System Id = file:/export/home/j2eetest/3.1.2/glassfish3/glassfish/domains/domain1/config/domain.xml Public Id = null Location Uri= file:/export/home/j2eetest/3.1.2/glassfish3/glassfish/domains/domain1/config/domain.xml CharacterOffset = 3634 The remote installation directory, /export/home/j2eetest/3.1.2/glassfish3, already exists. Use the --force option to overwrite it.

It is hard to see the actual cause of the problem. We should, 1. in the least print the last sentence on a line by itself but 2. ideally not include the information in between. Hence it would be:

1.
An error has occurred
Successfully connected to j2eetest@tuppy using keyfile /export/home/j2eetest/.ssh/id_rsa Command install-node-ssh failed. Ignoring unrecognized element schedules at Line number = 57 Column number = 18 System Id = file:/export/home/j2eetest/3.1.2/glassfish3/glassfish/domains/domain1/config/domain.xml Public Id = null Location Uri= file:/export/home/j2eetest/3.1.2/glassfish3/glassfish/domains/domain1/config/domain.xml CharacterOffset = 3425 Ignoring unrecognized element backup-configs at Line number = 62 Column number = 23 System Id = file:/export/home/j2eetest/3.1.2/glassfish3/glassfish/domains/domain1/config/domain.xml Public Id = null Location Uri= file:/export/home/j2eetest/3.1.2/glassfish3/glassfish/domains/domain1/config/domain.xml CharacterOffset = 3634

The remote installation directory, /export/home/j2eetest/3.1.2/glassfish3, already exists. Use the --force option to overwrite it.

2.
An error has occurred

Successfully connected to j2eetest@tuppy using keyfile /export/home/j2eetest/.ssh/id_rsa Command install-node-ssh failed.

The remote installation directory, /export/home/j2eetest/3.1.2/glassfish3, already exists. Use the --force option to overwrite it.

I'm attaching screenshot of the error message. I understand it is too late to fix this issue for this release, hence logging as minor. This issue surfaced after fixing http://java.net/jira/browse/GLASSFISH-18037.






[GLASSFISH-17723] restart-instance did not synchronize the pending changes. Created: 14/Nov/11  Updated: 06/Jun/12

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1.2_b09
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: easarina Assignee: Byron Nevins
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3x_regression

 Description   

Installed 3.1.2 Promoted b09 on two Solaris 10 Sparc machines. During the test executed the follow commands:
=============================================================
11/11/2011 18:09:45 EXIT: 0 asadmin --user admin --passwordfile /opt/appserver-sqe/ee/cluster_inf/password.txt stop-instance i101
11/11/2011 18:09:49 EXIT: 0 asadmin --user admin --passwordfile /opt/appserver-sqe/ee/cluster_inf/password.txt create-auth-realm --target c10 --classname com.sun.enterprise.security.auth.realm.certificate.CertificateRealm cert10
11/11/2011 18:10:14 EXIT: 0 asadmin --_auxinput - --interactive=false start-local-instance --node localhost-domain1 --sync none i101
11/11/2011 18:10:14 EXIT: 0 asadmin --user admin --passwordfile /opt/appserver-sqe/ee/cluster_inf/password.txt start-instance --sync none i101
11/11/2011 18:10:47 EXIT: 0 asadmin --host localhost --port 4848 --secure=false --terse=false --echo=false --interactive=false start-local-instance --verbose=false --debug=false --node localhost-domain1 i101
11/11/2011 18:10:49 EXIT: 0 asadm