Issue Details (XML | Word | Printable)

Key: GLASSFISH-11088
Type: Bug Bug
Status: Open Open
Priority: Major Major
Assignee: sakshi.jain
Reporter: bavik78
Votes: 1
Watchers: 4
Operations

If you were logged in you would be able to see more operations.
glassfish

[embedded] Stopping GFv3 leaves several daemon threads running

Created: 18/Nov/09 01:20 PM   Updated: 06/Mar/12 09:59 PM
Component/s: embedded
Affects Version/s: V3
Fix Version/s: not determined

Time Tracking:
Not Specified

File Attachments: 1. Java Source File GlassFish11088.java (3 kB) 01/Jun/11 01:19 AM - Bhavanishankar
2. File jstack (7 kB) 01/Jun/11 01:19 AM - Bhavanishankar

Environment:

Operating System: Windows XP
Platform: PC

Issue Links:
Dependency
 

Issuezilla Id: 11,088
Status Whiteboard:

v3_exclude

Tags: 3_1-exclude 3_1-next_release-note-added 3_1-next_release-notes 3_1_1-next 3_1_1-scrubbed
Participants: bavik78, Bhavanishankar, brdjcx, dochez, kumara, sakshi.jain, Satish Kumar, scatari, sirajg, Tim Quinn and Tom Mueller


 Description  « Hide

After stopping the server there is a bunch of daemon threads:
transaction-manager, AutoDeployer, DynamicReloader, pool-X-thread-Y.

Most of these threads (all except of some instances of pool-X-thread-Y) are
never finished until VM exit.
In the use case, when embedded GF is restarted many times during main
application session this can cause "Out-Of-Memory" problem and other side effects.



bavik78 added a comment - 18/Nov/09 01:21 PM

2 CC added


dochez added a comment - 18/Nov/09 01:41 PM

not a showstopper, few deamon threads hanging around after server.close()


kumara added a comment - 07/Dec/09 02:34 AM

Setting target release for unresolved issues submitted on v3 release to the next release. Not changing
issues submitted on v2.x release because they might not apply to v3.next release.


sirajg added a comment - 08/Jul/10 10:38 AM

transfer


sirajg added a comment - 09/Jul/10 03:29 PM

After a start, deploy, stop, I see these threads (Thread.getAllStackTraces) :
Keep-Alive-Timer
Reference Handler
pool-8-thread-1
pool-1-thread-1
pool-6-thread-1
pool-7-thread-1
pool-4-thread-1
DynamicReloader
pool-2-thread-1
transaction-manager
AutoDeployer
Signal Dispatcher
pool-10-thread-1
pool-9-thread-1
pool-5-thread-1
pool-3-thread-1


Bhavanishankar added a comment - 10/Oct/10 09:58 PM

This issue is reproducible in v3.1 as well.

This is a core issue in glassfish, not particularly an embedded issue. But the
embedded users will have the impact. Bunch of threads belonging to different
modules remain even after GlassFishRuntime.shutdown(), so we need to go through
each of them and see why they remain.

May not be able to fix this for v3.1, hence updating the keyword.


Bhavanishankar added a comment - 01/Jun/11 01:19 AM

Simple testcase.

Instructions to run the test case:

1. Compile it:

javac -g -cp $S1AS_HOME/lib/embedded/glassfish-embedded-static-shell.jar GlassFish11088.java

2. Run it

java -cp $S1AS_HOME/lib/embedded/glassfish-embedded-static-shell.jar:. GlassFish11088

3. When it prompts "GlassFish has been stopped and disposed. After taking jstack, press any key to quit this program", please take the jstack.

In the jstack you will see number of glassfish threads still running. (I have attached the jstack output that I got with this testcase)

Once this issue is fixed, there should not be any running glassfish threads in the jstack.


brdjcx added a comment - 07/Jun/11 02:18 PM

Bug two years old and "No work has yet been logged on this issue."?

May be tolerable on Unix/Mac but a stopper on Windows. Those stray processes hold files open that IDEs/maven want to delete (mvn clean). The workaround on Windows goes like this: stop GF, stop Netbeans, kill the strays with TaskManager, and start everything back up and wait, wait, wait. Dozens of times per hour. NOT good.


Tim Quinn added a comment - 08/Jun/11 09:09 AM - edited

partial fix checked into the 3.1.1 branch and the trunk. The rev for the 3.1.1 branch is

Project: glassfish
Repository: svn
Revision: 47365
Author: tjquinn
Date: 2011-06-08 16:02:35 UTC
Link:

Here is the full check-in log for the trunk check-in. The files changes and the diffs are the same for both check-ins.

Project: glassfish
Repository: svn
Revision: 47366
Author: tjquinn
Date: 2011-06-08 16:06:58 UTC
Link:

Log Message:
------------
Partial fix for 11088

The dynamic reloader and auto-deployer daemon threads were not stopped as part of shutdown. This caused problems during embedded use.

These changes correct where the "cancel requested" indicator is cleared.

Tests: autodeploy and "touch .reload" manual tests

Revisions:
----------
47366

Modified Paths:
---------------
trunk/v3/deployment/autodeploy/src/main/java/org/glassfish/deployment/autodeploy/AutoDeployService.java
trunk/v3/core/kernel/src/main/java/com/sun/enterprise/v3/server/DynamicReloadService.java
trunk/v3/core/kernel/src/main/java/com/sun/enterprise/v3/server/DynamicReloader.java
trunk/v3/deployment/autodeploy/src/main/java/org/glassfish/deployment/autodeploy/AutoDeployer.java


Bhavanishankar added a comment - 14/Jun/11 03:22 AM

Apart from the simple start/stop of embedded GlassFish, we will also have to check which threads keep running in the start/deploy/undeploy/stop scenario.

To do that, the existing devtest can be used like this:

1. cd v3/tests/embedded/cdi_ejb_jpa (in your 3.1.1 workspace).

2. Modify the test a bit so that it gives a chance to take the jstack/jmap before it exits, like this:

--- src/test/java/org/glassfish/tests/embedded/cdi_ejb_jpa/BasicCDITest.java    (revision 47440)
+++ src/test/java/org/glassfish/tests/embedded/cdi_ejb_jpa/BasicCDITest.java    (working copy)
@@ -105,6 +105,11 @@
 
         glassfish.dispose();
 
+               System.out.println("GlassFish has been stopped and disposed. " +
+                                   "After taking jstack, press any key to quit this program");
+
+                       new BufferedReader(new InputStreamReader(System.in)).readLine();
+
     }   
 
     private void get(String urlStr, String result) throws Exception {

3. Set S1AS_HOME to your 3.1.1 glassfish image (eg., /space/image/glassfish3/glassfish)

4. Execute the test : mvn -P run-with-static-shell clean verify

5. Once the test finishes and prompts "GlassFish has been stopped and disposed. After taking jstack, press any key to quit this program", please take the jstack/jmap of the "SurefireBooter" process.


Satish Kumar added a comment - 20/Jun/11 06:20 AM

Out of the threads mentioned in the jstack, the following have been resolved-

DynamicReloader
AutoDeployer

The following threads are started by the event notification framework (EventsImpl) and return eventually after notifying the event listeners. This is taking a bit of time 30-60 secs before they return. But these are not long running threads.

pool-2-thread-1
pool-3-thread-1
pool-4-thread-1
pool-5-thread-1

transaction-manager:
The jta module starts a timer service from JavaEETransactionManagerSimplified which is not stopped during appserver shutdown. I have created a separate issue to track this - 16873.


Satish Kumar added a comment - 21/Jun/11 03:22 AM

Linking this issue to GF 16873...


scatari added a comment - 25/Jun/11 07:17 PM

Pre-approving for 3.1.1 as this is needed for embedded.


Satish Kumar added a comment - 06/Jul/11 04:50 PM

marking this for the next release since this issue is still waiting for 16873 which is unlikely to get fixed for 3.1.1


Bhavanishankar added a comment - 01/Dec/11 05:42 AM

.


Tom Mueller added a comment - 06/Mar/12 09:59 PM

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.