[GLASSFISH-15503] [STRESS] JRockit: Deadlock observed from jstack with JMS PortMapper involved Created: 10/Jan/11  Updated: 19/Dec/16  Due: 18/Jan/11  Resolved: 07/Jun/13

Status: Resolved
Project: glassfish
Component/s: jms
Affects Version/s: 3.1_dev
Fix Version/s: 4.1

Type: Bug Priority: Major
Reporter: varunrupela Assignee: David Zhao
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File instance101-jstack.out    
Issue Links:
Dependency
blocks GLASSFISH-15425 [STRESS][umbrella] 24x7 RichAccess ru... Open
Status Whiteboard:

Waiting for JRocket fix for bugster issues: 7011219 and 7011216

Tags: 3_1-exclude, 3_1_1-scrubbed, 3_1_2-exclude, bj-reviewed-t3

 Description   

Details of the scenarios that shows this issue are in the parent issue: http://java.net/jira/browse/GLASSFISH-15425

Attaching the jstack.

We are not sure at this point if this is the root cause of all the issues mentioned on 15425.



 Comments   
Comment by Satish Kumar [ 11/Jan/11 ]

Nigel and I had detailed discussion on this issue, following is our analysis:

The root cause of this problem appears to be the following exception in which the "Grizzly proxy", after it has accepted a client connection in behalf of the portmapper, passes the SocketChannel to the port mapper itself.

java.lang.NoClassDefFoundError: java/lang/String
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
at java.lang.Class.getMethod0(Class.java:2670)
at java.lang.Class.getMethod(Class.java:1603)
at com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter.handleRequest(ActiveJmsResourceAdapter.java:2196)
at
com.sun.enterprise.v3.services.impl.ServiceInitializerHandler.onAcceptInterest(ServiceInitializerHandler.java:114)
at
com.sun.grizzly.SelectorHandlerRunner.handleSelectedKey(SelectorHandlerRunner.java:301)
at
com.sun.grizzly.SelectorHandlerRunner.handleSelectedKeys(SelectorHandlerRunner.java:263)
at
com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:200)
at
com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:132)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

We can only suggest that the NoClassDefFoundError is a JRockit bug (a similar NoClassDefFoundError occurs in a Shoal thread at about the same time).

The result of this exception will cause the MQ client to hang whilst reading data from the port mapper. The socket has been established but no data is being returned. As a result the application's call to createConnection() will block.

This is taking place whilst the connection pool is being enlarged. We notice that due to the connection pool holding various locks whilst this is taking place, all other connection pool activity (including closing connections) will block as well.

The JVM issue has been logged in bugster http://monaco.sfbay.sun.com/detail.jsf?cr=7011216.

There is not much we can do on this bug but to wait and watch until the JVM issue is resolved.

Comment by Scott Fordin [ 23/Mar/11 ]

Need more info to add issue to 3.1 Release Notes.

Comment by Chris Kasso [ 18/Apr/11 ]

Since JRockit is not supported in 3.1 I am removing this issue as a RN item.

Comment by Satish Kumar [ 24/May/11 ]

As mentioned in the status whiteboard and in the previous comment, is appears to be more of a JRockit issue. We are still waiting for the JRockit related bugster issues to be resolved- 7011219 and 7011216.

Comment by Satish Kumar [ 08/Jul/11 ]

Downgrading the priority of this issue to major since this is not critical for the 3.1.1 release. No further action is required here until the Jrocket issues are fixed.

Comment by David Zhao [ 07/Jun/13 ]

The JRockit defects were fixed in 28.2.2, so close this issue.

Generated at Thu Apr 27 14:14:50 UTC 2017 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.