glassfish
  1. glassfish
  2. GLASSFISH-18024

virtual network interfaces introduced by virtualization systems regress Glassfish 3.1.2 GMS auto selection of an appropriate network interface to use

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.1.2_b14
    • Fix Version/s: 4.0
    • Labels:
      None
    • Environment:

      OEL 5,
      JDK 1.6.0_24 64 bits

      Description

      The EJB automatic timer migration works in shutdown instance mode. That means after automatic timer is created, use asadmin stop-instance, the timer migration will happen. But it doesn't work in crashing mode when the instance containing timer was killed.

      The EJB timer app and logs are attached. The steps to reproduce the error are in EJB_Autotimer_FO/ant.output. The DAS log and instance logs are under EJB_Autotimer_FO/testListAutoTimer/logs/st-domain and st-cluster.

        Issue Links

          Activity

          Hide
          Joe Fialli added a comment -

          Due to XEN being installed on some machines that compose a cluster for this issue,
          not all machines are selecting network interfaces with matching characteristics.
          The XEN introduced network interfaces are IPv6 only and the machines in cluster without
          XEN have dual stack network interface for eth0. Thus, half machines are using IPv6 only
          and other half are using IPv4 addresses (as preferred for dual stack).

          Suggested workaround of create-cluster --gms-bind-interface-address eth0 hit the
          reported bug GLASSFISH-18047.

          Show
          Joe Fialli added a comment - Due to XEN being installed on some machines that compose a cluster for this issue, not all machines are selecting network interfaces with matching characteristics. The XEN introduced network interfaces are IPv6 only and the machines in cluster without XEN have dual stack network interface for eth0. Thus, half machines are using IPv6 only and other half are using IPv4 addresses (as preferred for dual stack). Suggested workaround of create-cluster --gms-bind-interface-address eth0 hit the reported bug GLASSFISH-18047 .
          Hide
          Joe Fialli added a comment - - edited

          Downgraded this issue to minor.

          Ming did confirm that when the machine with XEN installed was removed from the cluster,
          that the test did pass.

          A review of the server logs showed that not all instances were on same subnet.
          All instances that were on a machine with XEN installed on it were incorrectly
          selecting the XEN virtual network interface for GMS communications.

          Summary of issue:

          The introduction of non-multicast mode for Group Management Services (GMS) in Glassfish 3.1.2 altered which network interface was automatically selected to be used on a multi-homed machine for clustering communications. This change can result in some clustered instances
          no longer being able to join their running cluster.

          In Glassfish 3.1-3.1.1, a network interface that did not support multicast was not considered as a candidate to be selected as the network interface to be used for cluster communications.
          Thus, the automatic selection of network interfaces was impacted. Specifically,
          virtual network interfaces that used to be ignored since interface did not support multicast,
          have been incorrectly selected as the default network interfaces for cluster communications.

          Workarounds:

          • Either disable/remove the network interfaces that are being selected incorrectly.

          Or

          • Specify which network interface to use on the machine(s) selecting the incorrect network interface. Here is pointer to documentation on how to specify which network interface
            to use on a multi-home machine.

          Link: http://docs.oracle.com/cd/E18930_01/html/821-2426/gjfnl.html#gjdlw

          Show
          Joe Fialli added a comment - - edited Downgraded this issue to minor. Ming did confirm that when the machine with XEN installed was removed from the cluster, that the test did pass. A review of the server logs showed that not all instances were on same subnet. All instances that were on a machine with XEN installed on it were incorrectly selecting the XEN virtual network interface for GMS communications. Summary of issue: The introduction of non-multicast mode for Group Management Services (GMS) in Glassfish 3.1.2 altered which network interface was automatically selected to be used on a multi-homed machine for clustering communications. This change can result in some clustered instances no longer being able to join their running cluster. In Glassfish 3.1-3.1.1, a network interface that did not support multicast was not considered as a candidate to be selected as the network interface to be used for cluster communications. Thus, the automatic selection of network interfaces was impacted. Specifically, virtual network interfaces that used to be ignored since interface did not support multicast, have been incorrectly selected as the default network interfaces for cluster communications. Workarounds: Either disable/remove the network interfaces that are being selected incorrectly. Or Specify which network interface to use on the machine(s) selecting the incorrect network interface. Here is pointer to documentation on how to specify which network interface to use on a multi-home machine. Link: http://docs.oracle.com/cd/E18930_01/html/821-2426/gjfnl.html#gjdlw
          Hide
          Joe Fialli added a comment -

          Fix for this is integrated into shoal-1.6.18 (shoal svn 1745).

          Note that shoal 1.6.17 is in glassfish 3.1.2 so this is not fixed in Glassfish 3.1.2.

          Show
          Joe Fialli added a comment - Fix for this is integrated into shoal-1.6.18 (shoal svn 1745). Note that shoal 1.6.17 is in glassfish 3.1.2 so this is not fixed in Glassfish 3.1.2.
          Hide
          Tom Mueller added a comment -

          Is this fixed in 4.0 since shoal 1.6.18 is in 4.0?

          Show
          Tom Mueller added a comment - Is this fixed in 4.0 since shoal 1.6.18 is in 4.0?
          Hide
          Joe Fialli added a comment -

          It was quite a complex environment that this issue was reported against.
          We don not have automated test to verify such an environment.

          Here is detailed commit message on how this issue was addressed in the Shoal GMS.

          > Altered algorithm for selecting network interface. Unless java.net.preferIPv6Addresses is set to true,
          > will favor network interface supporting IPv4 and multicast. Will settle for network interface that
          > does not support multicast if one exists. Lastly will settle for network interface that does not
          > support preferred IPv address format.
          >
          > Fix for GLASSFISH-18047: allow a network interface name as BIND_INTERFACE_ADDRESS.
          > Allows one to set network interface such as "eth0" if all machines involved have same network interface name for a cluster.

          The issue has been marked resolved but it is unverified.

          Show
          Joe Fialli added a comment - It was quite a complex environment that this issue was reported against. We don not have automated test to verify such an environment. Here is detailed commit message on how this issue was addressed in the Shoal GMS. > Altered algorithm for selecting network interface. Unless java.net.preferIPv6Addresses is set to true, > will favor network interface supporting IPv4 and multicast. Will settle for network interface that > does not support multicast if one exists. Lastly will settle for network interface that does not > support preferred IPv address format. > > Fix for GLASSFISH-18047 : allow a network interface name as BIND_INTERFACE_ADDRESS. > Allows one to set network interface such as "eth0" if all machines involved have same network interface name for a cluster. The issue has been marked resolved but it is unverified.

            People

            • Assignee:
              Joe Fialli
              Reporter:
              mzh777
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: