glassfish
  1. glassfish
  2. GLASSFISH-15814

repeated FileInstall (autodeploy) threads started until Out of Memory error occurs

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 3.1_b40
    • Fix Version/s: None
    • Component/s: OSGi
    • Labels:
      None
    • Environment:

      Windows Vista 32Bit JDK 6.0.21

      Description

      When glassfish is started, it keeps starting a thread related to auto deployment until ultimately it runs out of memory. This thread dumps the following error:

      INFO: C:\util\java\glassfishv3\glassfish\domains\domain1\autodeploy\bundles does not exist, please create it.

      until the bundles directory is created and then it falls silent.

      On memory failure the following stack trace results:

      SEVERE: ERROR [org.osgi.service.cm.ManagedServiceFactory, id=77, bundle=76]: Unexpected problem updating Configuration PID=org.apache.felix.fileinstall.5ae705d0-886e-4dde-8f84-0f8640c29d1d, factoryPID=org.apache.felix.fileinstall, bundleLocation=file:/C:/util/java/glassfishv3/glassfish/modules/org.apache.felix.fileinstall.jar
      SEVERE: java.lang.OutOfMemoryError: unable to create new native thread
      at java.lang.Thread.start0(Native Method)
      at java.lang.Thread.start(Thread.java:597)
      at org.apache.felix.fileinstall.internal.DirectoryWatcher.start(DirectoryWatcher.java:208)
      at org.apache.felix.fileinstall.internal.FileInstall.updated(FileInstall.java:221)
      at org.apache.felix.fileinstall.internal.FileInstall$ConfigAdminSupport$Tracker.updated(FileInstall.java:339)
      at org.apache.felix.cm.impl.ConfigurationManager$UpdateConfiguration.run(ConfigurationManager.java:1460)
      at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:88)

        Issue Links

          Activity

          Hide
          jsl123 added a comment -

          I was unfortunately mistaken about the static nature of the number of threads. I'm not sure how I managed to get static thread counts as I've been unable to replicate it since.

          Let me know if you wish to discuss via email, etc rather than polluting the issue with comments...

          I've done another fresh install of 3.1-b41 and observe the same behaviour, in addition I've download the felix sources (guestimated the revision to match the glasfish build as best as I can) and stepped through the startup.

          It appears that the following behaviour is occurring:
          A DirectoryWatcher is created on ./modules/autostart which notices that the org.apache.felix.fileinstall-autodeploy-bundles.cfg file has been modified, this then indirectly creates a DirectoryWatcher on ./domains/domain1/autodeploy/bundles.
          The process is then repeated on the next loop in, however nothing external is modifying the config file. I'm not sure where in the felix code the config file is being modified...

          In addition in ConfigurationManager:ManagedServiceFactoryUpdate() the call to factoryPIDs returns an increasing number of PIDs on each execution of glassfish. It appears to be the sum of count of all DirectoryWatchers created above. For example if 10 were created on the first run and then 7 on the second, the number of PIDs is around 17. Which further exacerbates the problem by initialising N DirectoryWatchers on the bundles directory before the above process begins.

          Hope that helps

          Show
          jsl123 added a comment - I was unfortunately mistaken about the static nature of the number of threads. I'm not sure how I managed to get static thread counts as I've been unable to replicate it since. Let me know if you wish to discuss via email, etc rather than polluting the issue with comments... I've done another fresh install of 3.1-b41 and observe the same behaviour, in addition I've download the felix sources (guestimated the revision to match the glasfish build as best as I can) and stepped through the startup. It appears that the following behaviour is occurring: A DirectoryWatcher is created on ./modules/autostart which notices that the org.apache.felix.fileinstall-autodeploy-bundles.cfg file has been modified, this then indirectly creates a DirectoryWatcher on ./domains/domain1/autodeploy/bundles. The process is then repeated on the next loop in, however nothing external is modifying the config file. I'm not sure where in the felix code the config file is being modified... In addition in ConfigurationManager:ManagedServiceFactoryUpdate() the call to factoryPIDs returns an increasing number of PIDs on each execution of glassfish. It appears to be the sum of count of all DirectoryWatchers created above. For example if 10 were created on the first run and then 7 on the second, the number of PIDs is around 17. Which further exacerbates the problem by initialising N DirectoryWatchers on the bundles directory before the above process begins. Hope that helps
          Hide
          jsl123 added a comment -

          Hi, one more thing. Doing the same on Linux (gf 3.1 b41, ubuntu karmic, jdk6.18) results in 2 DirectorWatcher threads being created as expected (one on the modules/autostart directory and the second on autodeploy/bundles).

          The only difference seems to be is that the linux version is the Oracle release whereas the windows version is the Open Source release...

          Show
          jsl123 added a comment - Hi, one more thing. Doing the same on Linux (gf 3.1 b41, ubuntu karmic, jdk6.18) results in 2 DirectorWatcher threads being created as expected (one on the modules/autostart directory and the second on autodeploy/bundles). The only difference seems to be is that the linux version is the Oracle release whereas the windows version is the Open Source release...
          Hide
          jsl123 added a comment -

          This seems to be related to https://issues.apache.org/jira/browse/FELIX-1861

          No sure if this is part of the same thing, but, it looks like the list of valid PIDs isn't changed on exit hence the re-starting of extra threads on every restart of Glassfish.

          I suspect the "infinite" thread count is caused simply by my development environment restarting glassfish many times a day so apologies for not spotting that earlier.

          This is a killer for 2 reasons, the temp directory is polluted (I had 20k temp file-install directories) and and eventually the number of threads will exceed any OS limits.

          Show
          jsl123 added a comment - This seems to be related to https://issues.apache.org/jira/browse/FELIX-1861 No sure if this is part of the same thing, but, it looks like the list of valid PIDs isn't changed on exit hence the re-starting of extra threads on every restart of Glassfish. I suspect the "infinite" thread count is caused simply by my development environment restarting glassfish many times a day so apologies for not spotting that earlier. This is a killer for 2 reasons, the temp directory is polluted (I had 20k temp file-install directories) and and eventually the number of threads will exceed any OS limits.
          Hide
          Sanjeeb Sahoo added a comment -

          This is the same as the linked issue.

          Show
          Sanjeeb Sahoo added a comment - This is the same as the linked issue.
          Hide
          Sanjeeb Sahoo added a comment -

          Closing as a duplicate

          Show
          Sanjeeb Sahoo added a comment - Closing as a duplicate

            People

            • Assignee:
              Sanjeeb Sahoo
              Reporter:
              jsl123
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: