glassfish
  1. glassfish
  2. GLASSFISH-17195

GMS fails to initialize due to GMSException: can not find a first InetAddress

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Trivial Trivial
    • Resolution: Unresolved
    • Affects Version/s: 3.1.1
    • Fix Version/s: None
    • Labels:
      None
    • Environment:

      Windows 7, JDK 7, Wireless Network Interface, GlassFish 3.1.1

      Description

      Created a 2-instance cluster using GlassFish 3.1.1 on Windows 7/JDK7 and starting the cluster/instances gives the following error:

      [#|2011-08-16T10:59:17.116-0700|CONFIG|glassfish3.1.1|ShoalLogger|_ThreadID=1;_ThreadName=Thread-2;|
      GrizzlyNetworkManager Configuration
      BIND_INTERFACE_ADDRESS:null NetworkInterfaceName:null
      TCPSTARTPORT..TCPENDPORT:9090..9200
      MULTICAST_ADDRESS:MULTICAST_PORT:228.9.143.78:9635 MULTICAST_PACKET_SIZE:65536 MULTICAST_TIME_TO_LIV
      E: default
      FAILURE_DETECT_TCP_RETRANSMIT_TIMEOUT(ms):10000
      ThreadPool CORE_POOLSIZE:20 MAX_POOLSIZE:50 POOL_QUEUE_SIZE:4096 KEEP_ALIVE_TIME(ms):60000
      HIGH_WATER_MARK:1024 NUMBER_TO_RECLAIM:10 MAX_PARALLEL:15
      START_TIMEOUT(ms):15000 WRITE_TIMEOUT(ms):10000
      MAX_WRITE_SELECTOR_POOL_SIZE:30
      VIRTUAL_MULTICAST_URI_LIST:null

      #]

      [#|2011-08-16T10:59:17.157-0700|INFO|glassfish3.1.1|grizzly|_ThreadID=20;_ThreadName=Thread-2;|GRIZZ
      LY0001: Starting Grizzly Framework 1.9.36 - 8/16/11 10:59 AM|#]

      [#|2011-08-16T10:59:17.184-0700|CONFIG|glassfish3.1.1|ShoalLogger|_ThreadID=1;_ThreadName=Thread-2;|
      Grizzly controller listening on /0:0:0:0:0:0:0:0:9179. Controller started in 37 ms|#]

      [#|2011-08-16T10:59:17.422-0700|SEVERE|glassfish3.1.1|javax.org.glassfish.gms.org.glassfish.gms|_Thr
      eadID=1;_ThreadName=Thread-2;|GMSAD1017: GMS failed to start. See stack trace for additional informa
      tion.
      com.sun.enterprise.ee.cms.core.GMSException: failed to join group c1
      at com.sun.enterprise.ee.cms.impl.base.GMSContextImpl.join(GMSContextImpl.java:181)
      at com.sun.enterprise.ee.cms.impl.common.GroupManagementServiceImpl.join(GroupManagementServ
      iceImpl.java:382)
      at org.glassfish.gms.GMSAdapterImpl.initializeGMS(GMSAdapterImpl.java:576)
      at org.glassfish.gms.GMSAdapterImpl.initialize(GMSAdapterImpl.java:199)
      at org.glassfish.gms.bootstrap.GMSAdapterService.loadModule(GMSAdapterService.java:218)
      at org.glassfish.gms.bootstrap.GMSAdapterService.checkCluster(GMSAdapterService.java:192)
      at org.glassfish.gms.bootstrap.GMSAdapterService.postConstruct(GMSAdapterService.java:136)
      at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:131)
      at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:91)
      at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:82)
      at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67)
      at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:139)
      at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:76)
      at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:253)
      at com.sun.enterprise.v3.server.AppServerStartup.doStart(AppServerStartup.java:145)
      at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:136)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:6
      3)
      at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishImpl.start(OSGiGlassFishImpl.jav
      a:69)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:1
      17)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:601)
      at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97)
      at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55)
      Caused by: com.sun.enterprise.ee.cms.core.GMSException: initialization failure
      at com.sun.enterprise.mgmt.ClusterManager.<init>(ClusterManager.java:142)
      at com.sun.enterprise.ee.cms.impl.base.GroupCommunicationProviderImpl.initializeGroupCommuni
      cationProvider(GroupCommunicationProviderImpl.java:164)
      at com.sun.enterprise.ee.cms.impl.base.GMSContextImpl.join(GMSContextImpl.java:175)
      ... 25 more
      Caused by: java.io.IOException: can not find a first InetAddress
      at com.sun.enterprise.mgmt.transport.grizzly.GrizzlyNetworkManager.start(GrizzlyNetworkManag
      er.java:376)
      at com.sun.enterprise.mgmt.ClusterManager.<init>(ClusterManager.java:140)
      ... 27 more

      #]

      Here are the wireless network settings:

      D:\tools\glassfish\3.1.1\ose-glassfish3-full>ipconfig /all

      Windows IP Configuration

      Host Name . . . . . . . . . . . . : ARUNGUP-LAP
      Primary Dns Suffix . . . . . . . : st-users.us.oracle.com
      Node Type . . . . . . . . . . . . : Hybrid
      IP Routing Enabled. . . . . . . . : No
      WINS Proxy Enabled. . . . . . . . : No
      DNS Suffix Search List. . . . . . : st-users.us.oracle.com
      us.oracle.com

      Ethernet adapter Bluetooth Network Connection:

      Media State . . . . . . . . . . . : Media disconnected
      Connection-specific DNS Suffix . :
      Description . . . . . . . . . . . : Bluetooth Device (Personal Area Network)
      Physical Address. . . . . . . . . : 70-F1-A1-9B-D6-3C
      DHCP Enabled. . . . . . . . . . . : Yes
      Autoconfiguration Enabled . . . . : Yes

      Wireless LAN adapter Wireless Network Connection:

      Connection-specific DNS Suffix . : us.oracle.com
      Description . . . . . . . . . . . : Intel(R) Centrino(R) Advanced-N 6200 AGN
      Physical Address. . . . . . . . . : 00-27-10-17-FB-9C
      DHCP Enabled. . . . . . . . . . . : Yes
      Autoconfiguration Enabled . . . . : Yes
      IPv4 Address. . . . . . . . . . . : 10.151.1.82(Preferred)
      Subnet Mask . . . . . . . . . . . : 255.255.224.0
      Lease Obtained. . . . . . . . . . : Tuesday, August 16, 2011 9:18:51 AM
      Lease Expires . . . . . . . . . . : Tuesday, August 16, 2011 2:35:41 PM
      Default Gateway . . . . . . . . . : 10.151.0.1
      DHCP Server . . . . . . . . . . . : 10.196.255.250
      DNS Servers . . . . . . . . . . . : 148.87.1.22
      148.87.112.101
      NetBIOS over Tcpip. . . . . . . . : Enabled

      Ethernet adapter Local Area Connection:

      Media State . . . . . . . . . . . : Media disconnected
      Connection-specific DNS Suffix . : us.oracle.com
      Description . . . . . . . . . . . : Intel(R) 82577LM Gigabit Network Connection
      Physical Address. . . . . . . . . : 00-26-B9-F1-15-19
      DHCP Enabled. . . . . . . . . . . : Yes
      Autoconfiguration Enabled . . . . : Yes

      validate-multicast with/without --bindaddress passes.

      The cluster could be successfully started with the wired network.

      Tried the same steps on home wired/wireless network and got the same results.

        Activity

        Hide
        Joe Fialli added a comment -

        There is insufficient information to evaluate this issue. The submittd ipconfig output from windows is missing whether multicast is enabled or not for the network interface.

        GMS will not automatically select a network interface when NetworkInterface.supportsMulticast() does not
        return true.

        Please submit the output of running following command to confirm whether NetworkInterface.supportsMulticast()
        is returning true for the wireless network interface.

        cd to glassfish installation directory and run the following command:

        $ java -classpath glassfish3/glassfish/modules/shoal-gms-impl.jar com.sun.enterprise.mgmt.transport.NetworkUtility

        Here is the output from my mac from running this.

        AllLocalAddresses() = [/fe80:0:0:0:223:32ff:fe97:5cf7%5, /10.152.23.224, /fe80:0:0:0:0:0:0:1%1]
        getFirstNetworkInterface() = name:en0 (en0)
        getFirstInetAddress( true ) = /fe80:0:0:0:223:32ff:fe97:5cf7%5
        getFirstInetAddress( false ) = /10.152.23.224
        getFirstNetworkInteface() = name:en0 (en0)
        getFirstInetAddress(firstNetworkInteface, true) = /fe80:0:0:0:223:32ff:fe97:5cf7%5
        getFirstInetAddress(firstNetworkInteface, false) = /10.152.23.224

        The issue is that automatic selection of the network interface is failing, the above is the unit test for this case.
        Did you try the workaround of explicitly setting BIND_INTERFACE_ADDRESS ?

        Show
        Joe Fialli added a comment - There is insufficient information to evaluate this issue. The submittd ipconfig output from windows is missing whether multicast is enabled or not for the network interface. GMS will not automatically select a network interface when NetworkInterface.supportsMulticast() does not return true. Please submit the output of running following command to confirm whether NetworkInterface.supportsMulticast() is returning true for the wireless network interface. cd to glassfish installation directory and run the following command: $ java -classpath glassfish3/glassfish/modules/shoal-gms-impl.jar com.sun.enterprise.mgmt.transport.NetworkUtility Here is the output from my mac from running this. AllLocalAddresses() = [/fe80:0:0:0:223:32ff:fe97:5cf7%5, /10.152.23.224, /fe80:0:0:0:0:0:0:1%1] getFirstNetworkInterface() = name:en0 (en0) getFirstInetAddress( true ) = /fe80:0:0:0:223:32ff:fe97:5cf7%5 getFirstInetAddress( false ) = /10.152.23.224 getFirstNetworkInteface() = name:en0 (en0) getFirstInetAddress(firstNetworkInteface, true) = /fe80:0:0:0:223:32ff:fe97:5cf7%5 getFirstInetAddress(firstNetworkInteface, false) = /10.152.23.224 The issue is that automatic selection of the network interface is failing, the above is the unit test for this case. Did you try the workaround of explicitly setting BIND_INTERFACE_ADDRESS ?
        Hide
        Joe Fialli added a comment -

        lowered priority since explicitly setting BIND_INTERFACE_ADDRESS will work around the issue.
        (See link http://download.oracle.com/docs/cd/E18930_01/html/821-2426/gjfnl.html#gjdlw
        for details on how to configure this property.

        Additionally, while inconvenient that automatic selection of a network interface is not working properly,
        more info is needed to verify network interface configuration since the submitted info does
        not contain MULTICAST enabled info. The request for additional info in comments section will resolve
        the shortage of information and allow us to determine whether this is a truely a blocking issue
        for wireless networks on windows 7 using jdk 7.

        Show
        Joe Fialli added a comment - lowered priority since explicitly setting BIND_INTERFACE_ADDRESS will work around the issue. (See link http://download.oracle.com/docs/cd/E18930_01/html/821-2426/gjfnl.html#gjdlw for details on how to configure this property. Additionally, while inconvenient that automatic selection of a network interface is not working properly, more info is needed to verify network interface configuration since the submitted info does not contain MULTICAST enabled info. The request for additional info in comments section will resolve the shortage of information and allow us to determine whether this is a truely a blocking issue for wireless networks on windows 7 using jdk 7.
        Hide
        Joe Fialli added a comment -

        Awaiting confirmation from reporter if this issue is due to windows firewall and requires
        network configuration by user to enable network communications between processes.

        Specifically, create an inbound rule in Windows Firewall that allows all connections from all
        other members of the cluster.

        Show
        Joe Fialli added a comment - Awaiting confirmation from reporter if this issue is due to windows firewall and requires network configuration by user to enable network communications between processes. Specifically, create an inbound rule in Windows Firewall that allows all connections from all other members of the cluster.
        Hide
        arungupta added a comment -

        The output from the command is:

        D:\tools\glassfish\3.1.1\ose-glassfish3-full>java -classpath glassfish3\glassfish\modules\shoal-gms-impl.jar com.sun.enterprise.mgmt.transport.NetworkUtility
        AllLocalAddresses() = [/127.0.0.1, /0:0:0:0:0:0:0:1]
        getFirstNetworkInterface() = name:lo (Software Loopback Interface 1)
        getFirstInetAddress( true ) = null
        getFirstInetAddress( false ) = null
        getFirstNetworkInteface() = name:lo (Software Loopback Interface 1)
        getFirstInetAddress(firstNetworkInteface, true) = null
        getFirstInetAddress(firstNetworkInteface, false) = null

        Explicitly setting GMS-BIND-INTERFACE-ADDRESS-c1 as a system property is the workaround.

        The firewall rules will be required even if the DAS/instances are all on the local machine ?

        Show
        arungupta added a comment - The output from the command is: D:\tools\glassfish\3.1.1\ose-glassfish3-full>java -classpath glassfish3\glassfish\modules\shoal-gms-impl.jar com.sun.enterprise.mgmt.transport.NetworkUtility AllLocalAddresses() = [/127.0.0.1, /0:0:0:0:0:0:0:1] getFirstNetworkInterface() = name:lo (Software Loopback Interface 1) getFirstInetAddress( true ) = null getFirstInetAddress( false ) = null getFirstNetworkInteface() = name:lo (Software Loopback Interface 1) getFirstInetAddress(firstNetworkInteface, true) = null getFirstInetAddress(firstNetworkInteface, false) = null Explicitly setting GMS-BIND-INTERFACE-ADDRESS-c1 as a system property is the workaround. The firewall rules will be required even if the DAS/instances are all on the local machine ?
        Hide
        Joe Fialli added a comment -

        Attempted to recreate reported issue to determine if this is a general issue that all configurations
        of glassfish 3.1.1, windows 7 and jdk 7 would hit running using wireless network.

        Downloaded JDK 7 and GlassFish 3.1.1 to a HP Windows 7 Professional Laptop with only wireless network connection.
        (This laptop was running Norton Security instead of MacAfee Security in this report.)

        Was able to create a GlassFish cluster and instances were able to see each other.
        Key Log info.

        Aug 18, 2011 7:36:01 AM com.sun.enterprise.admin.launcher.GFLauncherLogger info
        INFO: JVM invocation command line:
        C:\Program Files\Java\jdk1.7.0\bin\java.exe

        #|2011-08-18T07:37:21.524-0400|INFO|glassfish3.1.1|ShoalLogger|_ThreadID=18;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: mycluster : Members in view for JOINED_AND_READY_EVENT(before change analysis) are :
        1: MemberId: instance01, MemberType: CORE, Address: 10.0.1.11:9145:228.9.29.50:30647:mycluster:instance01
        2: MemberId: instance02, MemberType: CORE, Address: 10.0.1.11:9122:228.9.29.50:30647:mycluster:instance02
        3: MemberId: server, MemberType: SPECTATOR, Address: 10.0.1.11:9116:228.9.29.50:30647:mycluster:server

        #]

        Additionally, ran both ipconfig and ListNetsEx program that uses java.net.NetworkInterface methods used by
        GMS to locate the first inet address. Attaching complete output, here is key output from those two commands.

        Wireless LAN adapter Wireless Network Connection:

        Connection-specific DNS Suffix . : hsd1.ma.comcast.net.
        Description . . . . . . . . . . . : Atheros AR9285 802.11b/g/n WiFi Adapter
        Physical Address. . . . . . . . . : C4-17-FE-2C-6C-51
        DHCP Enabled. . . . . . . . . . . : Yes
        Autoconfiguration Enabled . . . . : Yes
        Link-local IPv6 Address . . . . . : fe80::4839:4e5d:143c:ea11%11(Preferred)
        IPv4 Address. . . . . . . . . . . : 10.0.1.11(Preferred)
        Subnet Mask . . . . . . . . . . . : 255.255.255.0
        Lease Obtained. . . . . . . . . . : Monday, August 15, 2011 9:41:36 AM
        Lease Expires . . . . . . . . . . : Monday, August 22, 2011 6:30:28 AM
        Default Gateway . . . . . . . . . : 10.0.1.1
        DHCP Server . . . . . . . . . . . : 10.0.1.1
        DHCPv6 IAID . . . . . . . . . . . : 314841086
        DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-EB-25-F6-00-26-9E-EC-3C-66
        DNS Servers . . . . . . . . . . . : 10.0.1.1
        NetBIOS over Tcpip. . . . . . . . : Enabled

        ListNetsEx output:
        Display name: Atheros AR9285 802.11b/g/n WiFi Adapter
        Name: net3
        InetAddress: /10.0.1.11
        InetAddress: /fe80:0:0:0:4839:4e5d:143c:ea11%11
        Up? true
        Loopback? false
        PointToPoint? false
        Supports multicast? true
        Virtual? false
        Hardware address: [-60, 23, -2, 44, 108, 81]
        MTU: 1500

        Note that unlike submitted case, that the network interface "up" is returning true, which allows automatic
        detection of inet address to be found.

        There is something about the submitter's configuration that is resulting in java.net.NetworkInterface.isUp()
        to incorrectly return false for the wireless adapter. Can not be sure what it is since was unable to recreate the
        failure. Downgrading this issue to trivial since there is a workaround and we were not able to recreate
        the failure with the provided configuration information.

        Show
        Joe Fialli added a comment - Attempted to recreate reported issue to determine if this is a general issue that all configurations of glassfish 3.1.1, windows 7 and jdk 7 would hit running using wireless network. Downloaded JDK 7 and GlassFish 3.1.1 to a HP Windows 7 Professional Laptop with only wireless network connection. (This laptop was running Norton Security instead of MacAfee Security in this report.) Was able to create a GlassFish cluster and instances were able to see each other. Key Log info. Aug 18, 2011 7:36:01 AM com.sun.enterprise.admin.launcher.GFLauncherLogger info INFO: JVM invocation command line: C:\Program Files\Java\jdk1.7.0\bin\java.exe #|2011-08-18T07:37:21.524-0400|INFO|glassfish3.1.1|ShoalLogger|_ThreadID=18;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: mycluster : Members in view for JOINED_AND_READY_EVENT(before change analysis) are : 1: MemberId: instance01, MemberType: CORE, Address: 10.0.1.11:9145:228.9.29.50:30647:mycluster:instance01 2: MemberId: instance02, MemberType: CORE, Address: 10.0.1.11:9122:228.9.29.50:30647:mycluster:instance02 3: MemberId: server, MemberType: SPECTATOR, Address: 10.0.1.11:9116:228.9.29.50:30647:mycluster:server #] Additionally, ran both ipconfig and ListNetsEx program that uses java.net.NetworkInterface methods used by GMS to locate the first inet address. Attaching complete output, here is key output from those two commands. Wireless LAN adapter Wireless Network Connection: Connection-specific DNS Suffix . : hsd1.ma.comcast.net. Description . . . . . . . . . . . : Atheros AR9285 802.11b/g/n WiFi Adapter Physical Address. . . . . . . . . : C4-17-FE-2C-6C-51 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::4839:4e5d:143c:ea11%11(Preferred) IPv4 Address. . . . . . . . . . . : 10.0.1.11(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Lease Obtained. . . . . . . . . . : Monday, August 15, 2011 9:41:36 AM Lease Expires . . . . . . . . . . : Monday, August 22, 2011 6:30:28 AM Default Gateway . . . . . . . . . : 10.0.1.1 DHCP Server . . . . . . . . . . . : 10.0.1.1 DHCPv6 IAID . . . . . . . . . . . : 314841086 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-EB-25-F6-00-26-9E-EC-3C-66 DNS Servers . . . . . . . . . . . : 10.0.1.1 NetBIOS over Tcpip. . . . . . . . : Enabled ListNetsEx output: Display name: Atheros AR9285 802.11b/g/n WiFi Adapter Name: net3 InetAddress: /10.0.1.11 InetAddress: /fe80:0:0:0:4839:4e5d:143c:ea11%11 Up? true Loopback? false PointToPoint? false Supports multicast? true Virtual? false Hardware address: [-60, 23, -2, 44, 108, 81] MTU: 1500 Note that unlike submitted case, that the network interface "up" is returning true, which allows automatic detection of inet address to be found. There is something about the submitter's configuration that is resulting in java.net.NetworkInterface.isUp() to incorrectly return false for the wireless adapter. Can not be sure what it is since was unable to recreate the failure. Downgrading this issue to trivial since there is a workaround and we were not able to recreate the failure with the provided configuration information.

          People

          • Assignee:
            Joe Fialli
            Reporter:
            arungupta
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated: