glassfish
  1. glassfish
  2. GLASSFISH-18360

Update tool configuration breaks default network behaviour

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: 3.1.1_b12
    • Fix Version/s: None
    • Component/s: update_center
    • Labels:
      None
    • Environment:

      Linux debian squeeze (6.0.4). Running my web applications in the same jvm as the DAS

      Description

      The update tool creates at some point a .updatetool directory with a init.cfg file.
      The init.cfg file contains (after default installation and executing latest updates)

      [main]
      date: 1328646278464
      optin.update.notification: true
      image_list: /home/glassfish_prod/glassfish312/bin/..
      [network]
      proxy.use.system: true
      proxy.required: false

      The setting

      proxy.use.system: true

      Is used by the class

      com.sun.pkg.client.SystemInfo

      to set the following system property

      java.net.useSystemProxies=true

      This class is executed when glassfish starts up.

      This setting overrides the setting in $

      {JAVA_HOME}/jre/lib/net.properties

      The actual problem I'm having is that the jvm (read glassfish) will crash when I
      excecute several threads which call an external web service.

      The crash dump shows (partial)

      Stack: [0x00007fd589d79000,0x00007fd589e7a000], sp=0x00007fd589e71578, free space=993k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      C [libdbus-1.so.3+0x28de0] dbus_malloc+0xa0

      Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
      j sun.net.spi.DefaultProxySelector.getSystemProxy(Ljava/lang/String;Ljava/lang/String;)Ljava/net/Proxy;+0
      j sun.net.spi.DefaultProxySelector.access00(Lsun/net/spi/DefaultProxySelector;Ljava/lang/String;Ljava/lang/String;)Ljava/net/Proxy;+3
      j sun.net.spi.DefaultProxySelector.run()Ljava/net/Proxy;+151
      j sun.net.spi.DefaultProxySelector.run()Ljava/lang/Object;+1
      v ~StubRoutines::call_stub
      J java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)Ljava/lang/Object;
      j sun.net.spi.DefaultProxySelector.select(Ljava/net/URI;)Ljava/util/List;+223
      j sun.net.www.protocol.http.HttpURLConnection.plainConnect()V+314
      j sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect()V+9
      j sun.net.www.protocol.http.HttpURLConnection.getOutputStream()Ljava/io/OutputStream;+134
      j sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream()Ljava/io/OutputStream;+4
      j com.sun.xml.ws.mex.client.HttpPoster.post(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Ljava/io/InputStream;+60

      The DefaultProxySelector (see top of stack) contains the following static block

      static {
      final String key = "java.net.useSystemProxies";
      Boolean b = (Boolean) AccessController.doPrivileged(
      new PrivilegedAction() {
      public Object run() {
      return NetProperties.getBoolean(key);
      }});
      if (b != null && b.booleanValue()) { java.security.AccessController.doPrivileged( new sun.security.action.LoadLibraryAction("net")); hasSystemProxies = init(); }

      This will set the hasSystemProxies boolean only when the java.net.useSystemProxies system property is true.
      The hasSystemProxies boolean is checked in the select method and only if it is true it will call the getSystemProxy
      method which will crash the jvm (when multiple threads do so).

      So actually, there probably is a concurrency bug/problem with the libdbus native library which causes the actual crash.

      But: I expect the system proxy not to be checked (as specified by ${JAVA_HOME}

      /jre/lib/net.properties)
      Update tool is being initialized when glassfish starts up and the proxy.use.system=true setting will cause
      the system property java.net.useSystemProxies to be set to true which effectively causes the problem.

        Activity

        Hide
        Bobby Bissett added a comment -

        Moving to 'update center' category. I know the categories are confusing, but 'upgrade tool' is for asupgrade which acts on a domain so that an older domain can be used with a newer GF installation.

        Show
        Bobby Bissett added a comment - Moving to 'update center' category. I know the categories are confusing, but 'upgrade tool' is for asupgrade which acts on a domain so that an older domain can be used with a newer GF installation.
        Hide
        Joe Di Pol added a comment -

        Too late for 3.1.2. Tagging to revisit for next release.

        Show
        Joe Di Pol added a comment - Too late for 3.1.2. Tagging to revisit for next release.
        Hide
        kovica added a comment -

        As a workaround you could make java use direct connection instead of trying system proxies.
        Do this:

        • cd $GLASSFISH_INSTALL_DIR/bin
        • edit file pkg and search for line
          echo "proxy.use.system=true" >> "$BOOTSTRAPPROPS"
        • change it to:
          echo "proxy.use.system=false" >> "$BOOTSTRAPPROPS"

        Do the same with updatetool file if you want to install the graphical GUI to update tool

        Show
        kovica added a comment - As a workaround you could make java use direct connection instead of trying system proxies. Do this: cd $GLASSFISH_INSTALL_DIR/bin edit file pkg and search for line echo "proxy.use.system=true" >> "$BOOTSTRAPPROPS" change it to: echo "proxy.use.system=false" >> "$BOOTSTRAPPROPS" Do the same with updatetool file if you want to install the graphical GUI to update tool
        Hide
        thehpi added a comment -

        The pkg file does not contain this line. The pkg.bat file however does.
        I did find this line in the file $GLASSFISH_INSTALL_DIR/pkg/lib/pkg-bootstub.sh
        So it looks like this is the file that needs the fix.

        Show
        thehpi added a comment - The pkg file does not contain this line. The pkg.bat file however does. I did find this line in the file $GLASSFISH_INSTALL_DIR/pkg/lib/pkg-bootstub.sh So it looks like this is the file that needs the fix.
        Hide
        kovica added a comment -

        Sorry, I don't point out that I'm using glassfish-3.1.2.zip as the installation file.

        Show
        kovica added a comment - Sorry, I don't point out that I'm using glassfish-3.1.2.zip as the installation file.
        Hide
        ljnelson added a comment - - edited

        Hello; this bug appears to be related to research done as part of https://java.net/jira/browse/GLASSFISH-12213. It affects GlassFish 3.1.2.2 as well.

        Notes for posterity and non-GlassFish-team people:

        The Java pkg client has a class called com.sun.pkg.client.SystemInfo. At some point GlassFish must cause this class to get loaded at startup.

        When this class loads, it ends up having its loadProxyInfo() method called.

        This method looks for either an init.cfg or a defaults.cfg file in various locations. One of those locations is (on my Mac, as an example) /Users/ljnelson/Library/Application Support/updatetool.

        If it finds such a file, and if that file has a property setting of:

        proxy.use.system = true
        

        ...then SystemInfo will call System.setProperty("java.net.useSystemProxies", "true").

        This has a couple of side effects.

        The first side effect is that it overrides whatever is present in $JAVA_HOME/jre/lib/net.properties in its java.net.useSystemProxies line. (Actually it's not clear to me this file is consulted under GlassFish, as sun.net.NetProperties constructs the path to that file by doing something basically identical to System.getProperty("java.home") + "/lib/net.properties", which at least on my Mac would not involve the jre subdirectory.) So the value of the java.net.useSystemProxies property is essentially always true when GlassFish is running.

        Next, this has the effect of triggering a non-threadsafe native library load and execution by way of sun.net.spi.DefaultProxySelector, resulting in https://java.net/jira/browse/GLASSFISH-12213 (the potential of a JVM crash). Briefly, since DefaultProxySelector concludes that system proxies are to be used, it makes use of libgconf which turns out to be at the root of this JVM bug: http://bugs.sun.com/view_bug.do?bug_id=7188755 This JVM issue is fixed in a later build of Java 7.

        The workaround for this bug (maybe?) and GLASSFISH-12213 is most likely to make sure that defaults.cfg or init.cfg contains the line:

        proxy.use.system = false
        

        ,,,as that will avoid the System.setProperty() call in SystemInfo and hence the native library load.

        Show
        ljnelson added a comment - - edited Hello; this bug appears to be related to research done as part of https://java.net/jira/browse/GLASSFISH-12213 . It affects GlassFish 3.1.2.2 as well. Notes for posterity and non-GlassFish-team people: The Java pkg client has a class called com.sun.pkg.client.SystemInfo . At some point GlassFish must cause this class to get loaded at startup. When this class loads, it ends up having its loadProxyInfo() method called. This method looks for either an init.cfg or a defaults.cfg file in various locations. One of those locations is (on my Mac, as an example) /Users/ljnelson/Library/Application Support/updatetool . If it finds such a file, and if that file has a property setting of: proxy.use.system = true ...then SystemInfo will call System.setProperty("java.net.useSystemProxies", "true") . This has a couple of side effects. The first side effect is that it overrides whatever is present in $JAVA_HOME/jre/lib/net.properties in its java.net.useSystemProxies line. (Actually it's not clear to me this file is consulted under GlassFish, as sun.net.NetProperties constructs the path to that file by doing something basically identical to System.getProperty("java.home") + "/lib/net.properties" , which at least on my Mac would not involve the jre subdirectory.) So the value of the java.net.useSystemProxies property is essentially always true when GlassFish is running. Next, this has the effect of triggering a non-threadsafe native library load and execution by way of sun.net.spi.DefaultProxySelector , resulting in https://java.net/jira/browse/GLASSFISH-12213 (the potential of a JVM crash). Briefly, since DefaultProxySelector concludes that system proxies are to be used, it makes use of libgconf which turns out to be at the root of this JVM bug: http://bugs.sun.com/view_bug.do?bug_id=7188755 This JVM issue is fixed in a later build of Java 7. The workaround for this bug (maybe?) and GLASSFISH-12213 is most likely to make sure that defaults.cfg or init.cfg contains the line: proxy.use.system = false ,,,as that will avoid the System.setProperty() call in SystemInfo and hence the native library load.

          People

          • Assignee:
            Snjezana Sevo-Zenzerovic
            Reporter:
            thehpi
          • Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated: