glassfish
  1. glassfish
  2. GLASSFISH-17458

in non-multicast mode, one failed to connect per cluster instance at startup

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.1.2_b05
    • Fix Version/s: not determined
    • Labels:
      None
    • Environment:

      Linux 2.6.18-164.0.0.0.1.el5

      Description

      Glassfish version 3.1.2 build 5

      Lots of the following exception are being seen the the server logs.

      Expected: All exceptions to be handled

      http://aras2.us.oracle.com:8080/logs/gf31/gms//set_10_20_11_t_11_56_02/scenario_0012_Thu_Oct_20_12_00_37_PDT_2011.html

      11-10-20T18:56:57.888+0000|INFO|glassfish3.1.2|ShoalLogger.nomcast|_ThreadID=84;_ThreadName=Thread-2;|failed to send message to a virtual multicast endpoint[10.133.184.137:9090:230.30.1.1:9090:clusterz1:Unknown_10.133.184.137_9090] message=[MessageImpl[v1:MASTER_NODE_MESSAGE: NAD, Target: 10.133.184.137:9090:230.30.1.1:9090:clusterz1:Unknown_10.133.184.137_9090 , Source: 10.133.184.207:9090:230.30.1.1:9090:clusterz1:server, MQ, ]
      java.io.IOException: failed to connect to 10.133.184.137:9090:230.30.1.1:9090:clusterz1:Unknown_10.133.184.137_9090
      at com.sun.enterprise.mgmt.transport.grizzly.grizzly1_9.GrizzlyTCPConnectorWrapper.send(GrizzlyTCPConnectorWrapper.java:132)
      at com.sun.enterprise.mgmt.transport.grizzly.grizzly1_9.GrizzlyTCPConnectorWrapper.doSend(GrizzlyTCPConnectorWrapper.java:96)
      at com.sun.enterprise.mgmt.transport.AbstractMessageSender.send(AbstractMessageSender.java:74)
      at com.sun.enterprise.mgmt.transport.VirtualMulticastSender.doBroadcast(VirtualMulticastSender.java:134)
      at com.sun.enterprise.mgmt.transport.AbstractMulticastMessageSender.broadcast(AbstractMulticastMessageSender.java:70)
      at com.sun.enterprise.mgmt.transport.grizzly.GrizzlyNetworkManager.broadcast(GrizzlyNetworkManager.java:295)
      at com.sun.enterprise.mgmt.MasterNode.send(MasterNode.java:1338)
      at com.sun.enterprise.mgmt.MasterNode.discoverMaster(MasterNode.java:382)
      at com.sun.enterprise.mgmt.MasterNode.startMasterNodeDiscovery(MasterNode.java:1235)
      at com.sun.enterprise.mgmt.MasterNode.run(MasterNode.java:1204)
      at java.lang.Thread.run(Thread.java:722)
      Caused by: java.net.ConnectException: Connection refused
      at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
      at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
      at com.sun.grizzly.TCPConnectorHandler.finishConnect(TCPConnectorHandler.java:297)
      at com.sun.grizzly.connectioncache.client.CacheableConnectorHandler.finishConnect(CacheableConnectorHandler.java:230)
      at com.sun.enterprise.mgmt.transport.grizzly.grizzly1_9.GrizzlyTCPConnectorWrapper$CloseControlCallbackHandler.onConnect(GrizzlyTCPConnectorWrapper.java:185)
      at com.sun.grizzly.CallbackHandlerContextTask.doCall(CallbackHandlerContextTask.java:70)
      at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
      at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
      at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
      at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
      ... 1 more

      #]

        Activity

        Hide
        Tom Mueller added a comment -

        Bulk update to set Fix Version to "not determined" for issues that had it set to a version that has already been released.

        Show
        Tom Mueller added a comment - Bulk update to set Fix Version to "not determined" for issues that had it set to a version that has already been released.
        Hide
        Joe Fialli added a comment -

        This is only occuring in Glassfish Shoal QE test when running with GMS_DISCOVERY_URI_LIST set to
        a list on instances that have not been created (or started yet) and the DAS initially joins the cluster.

        There is only one exception per cluster member listed in GMS_DISCOVERY_URI_LIST.
        For the test case this is reported against, there are 9 instances, so there are nine connection
        failed when DAS joins cluster initially and those instances have yet to been created and started.
        When DAS first joins cluster and no instance has even been created yet,
        the DISCOVERY_URI_LIST contains connection info to yet to be created instances.

        We will demote the failed connections during discovery from WARNING to FINE, this
        will enable us to debug network configuration issues (such as firewalls) without
        the nusance of always seeing one failure per cluster member referenced in GMS_DISCOVERY_URI_LIST.

        Note: this issue does not apply to GMS_DISCOVERY_URI_LIST set to "generate" or to group discovery
        via UDP multicast.

        Show
        Joe Fialli added a comment - This is only occuring in Glassfish Shoal QE test when running with GMS_DISCOVERY_URI_LIST set to a list on instances that have not been created (or started yet) and the DAS initially joins the cluster. There is only one exception per cluster member listed in GMS_DISCOVERY_URI_LIST. For the test case this is reported against, there are 9 instances, so there are nine connection failed when DAS joins cluster initially and those instances have yet to been created and started. When DAS first joins cluster and no instance has even been created yet, the DISCOVERY_URI_LIST contains connection info to yet to be created instances. We will demote the failed connections during discovery from WARNING to FINE, this will enable us to debug network configuration issues (such as firewalls) without the nusance of always seeing one failure per cluster member referenced in GMS_DISCOVERY_URI_LIST. Note: this issue does not apply to GMS_DISCOVERY_URI_LIST set to "generate" or to group discovery via UDP multicast.

          People

          • Assignee:
            Joe Fialli
            Reporter:
            zorro
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated: