Issue Details (XML | Word | Printable)

Key: GLASSFISH-20924
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Critical Critical
Assignee: Jennifer Chou
Reporter: Joe Di Pol
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
glassfish

ConcurrentModificationException in StatsProviderManager.unregister() at startup

Created: 11/Dec/13 06:04 PM   Updated: 16/Dec/13 04:02 PM   Resolved: 16/Dec/13 04:02 PM
Component/s: monitoring
Affects Version/s: 4.0
Fix Version/s: 4.0.1

Time Tracking:
Not Specified

File Attachments: 1. Text File diff.txt (2 kB) 11/Dec/13 06:21 PM - Jennifer Chou
2. Java Source File StatsProviderManager.java (5 kB) 11/Dec/13 06:21 PM - Jennifer Chou

Issue Links:
Dependency
 

Tags:
Participants: Jennifer Chou, Joe Di Pol and oleksiys


 Description  « Hide

On large scale GlassFish cluster testing we sometimes see the following exception on GlassFish instance startup resulting in the instance not coming up:

[#|2013-12-04T20:20:41.483+0000|SEVERE|glassfish 4.0|javax.enterprise.system.core|_ThreadID=1;_ThreadName=main;_TimeMillis=1386188441483;_LevelValue=1000;_MessageID=NCLS-CORE-00019;|
  Shutting down server due to startup exception
java.util.ConcurrentModificationException
    at java.util.Vector$Itr.checkForComodification(Vector.java:1156)
    at java.util.Vector$Itr.next(Vector.java:1133)
    at org.glassfish.external.probe.provider.StatsProviderManager.unregister(StatsProviderManager.java:100)
    at com.sun.enterprise.v3.services.impl.monitor.GrizzlyMonitoring.registerThreadPoolStatsProvider(GrizzlyMonitoring.java:145)
    at com.sun.enterprise.v3.services.impl.GlassfishNetworkListener.registerMonitoringStatsProviders(GlassfishNetworkListener.java:289)
    at com.sun.enterprise.v3.services.impl.GlassfishNetworkListener.start(GlassfishNetworkListener.java:107)
    at com.sun.enterprise.v3.services.impl.GrizzlyProxy.start0(GrizzlyProxy.java:267)
    at com.sun.enterprise.v3.services.impl.GrizzlyProxy.start(GrizzlyProxy.java:241)
    at com.sun.enterprise.v3.services.impl.GrizzlyService.createNetworkProxy(GrizzlyService.java:567)
    at com.sun.enterprise.v3.services.impl.GrizzlyService.postConstruct(GrizzlyService.java:490)
    at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:298)
    at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:346)
    at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
    at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:225)
    at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:82)
    at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2358)
    at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:98)
    at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:87)
    at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1162)
    at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1147)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744) 


Jennifer Chou added a comment - 11/Dec/13 06:21 PM

Synchronize the methods on StatsProviderManager that update the Vector toBeRegistered: registerStatsProvider, unregister,and setStatsProviderManagerDelegate. Hopefully, performance will not be an issue since these methods are very small.


oleksiys added a comment - 11/Dec/13 09:22 PM

Jennifer, pls. change Vector to ArrayList. The Vector class is thread-safe itself (its methods are synchronized), but as we synchronize StatsProviderManager methods - we don't need extra synchronization.


Jennifer Chou added a comment - 11/Dec/13 10:13 PM

Changed Vector to ArrayList.


Jennifer Chou added a comment - 16/Dec/13 04:02 PM

Integrated management-api-3.2.1-b002 which contains the fix. This should fix the ConcurrentModificationException. If the problem is still seen in the large scale cluster test, please reopen this issue.

r63025 | jc129909 | 2013-12-16 10:52:43 -0500 (Mon, 16 Dec 2013) | 4 lines

Integrate management-api-3.2.1-b002 which contains the fix for GLASSFISH-20924.
Approved by Romain Grecourt.
Passed QL, admin devtests.

Index: pom.xml
===================================================================
— pom.xml (revision 63024)
+++ pom.xml (working copy)
@@ -161,7 +161,7 @@
<jax-rs-api.version>2.0</jax-rs-api.version>
<mimepull.version>1.9.3</mimepull.version>
<jbi.version>1.0</jbi.version>

  • <glassfish-management-api.version>3.2.0-b001</glassfish-management-api.version>
    + <glassfish-management-api.version>3.2.1-b002</glassfish-management-api.version>
    <btrace.version>1.0.5</btrace.version>
    <opendmk.version>1.0-b01-ea</opendmk.version>
    <v3-docs.version>4.0-b27</v3-docs.version>