glassfish
  1. glassfish
  2. GLASSFISH-9497

Failure in instance startup when agent's JMX Connector port conflicts with instance port

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: v2.1.1
    • Fix Version/s: 3.1
    • Component/s: admin
    • Labels:
      None
    • Environment:

      Operating System: All
      Platform: Sun

    • Issuezilla Id:
      9,497
    • Status Whiteboard:
      Hide

      v3_exclude

      Show
      v3_exclude

      Description

      GlassFish V2.1.1 B31a Cluster Profile
      JDK 1.5.0_20
      Solaris 10 Sparc

      After configuring a 4 instance cluster, starting up cluster fails with the
      following port conflict issue in 1st instance. As a result, instance 1 fails to
      startup. This is my first time encountering this issue in V2.1.1. This looks
      like a corner case.

      After some investigation, found out that agent1 on instance1 machine uses port
      33820 as the JMX Connector port when agent is created. Whereas the other 3
      agents on the other 3 instance machines have different ports which doesn't
      conflict with instance's ports. By default, port 33820 is used for IIOP SSL
      Listener Port for the instances.

      I deleted agent1 and recreated it again. This time, it picked up a random port
      and not 33820. After this, the instance startups fine without any issues.

      We would need to fix this so that the jmx connector ports doesn't use any of the
      default ports used by the instances. Otherwise, users will also end up in such a
      situation.

      I am filing this under admin for now. Please change it if it is supposed to be
      in a different subcomponent.

      Exception in server log:
      ************************

      [#|2009-09-12T23:26:58.451-0700|INFO|sun-appserver2.1|javax.enterprise.system.c
      ontainer.web|_ThreadID=10;_ThreadName=main;com.ericsson.ssa.config.ConvergedCon
      textImpl;|REgistering Custom Context|#]

      [#|2009-09-12T23:27:03.253-0700|SEVERE|sun-appserver2.1|javax.enterprise.resour
      ce.corba|_ThreadID=10;_ThreadName=main;SSL;33820;_RequestID=9340f5b1-fc93-4e40-
      abeb-b528e180af5b;|IOP5006: Exception creating socket: [SSL, 33820]. Please ma
      ke sure the port is not being used.|#]

      [#|2009-09-12T23:27:03.271-0700|SEVERE|sun-appserver2.1|javax.enterprise.resour
      ce.corba|_ThreadID=10;_ThreadName=main;_RequestID=9340f5b1-fc93-4e40-abeb-b528e
      180af5b;|
      java.net.BindException: Address already in use
      at java.net.PlainSocketImpl.socketBind(Native Method)
      at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
      at java.net.ServerSocket.bind(ServerSocket.java:319)
      at java.net.ServerSocket.<init>(ServerSocket.java:185)
      at javax.net.ssl.SSLServerSocket.<init>(SSLServerSocket.java:106)
      at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.<init>(SSLServerSoc
      ketImpl.java:106)
      at com.sun.net.ssl.internal.ssl.SSLServerSocketFactoryImpl.createServer
      Socket(SSLServerSocketFactoryImpl.java:71)
      at com.sun.enterprise.iiop.IIOPSSLSocketFactory.createSSLServerSocket(I
      IOPSSLSocketFactory.java:406)
      at com.sun.enterprise.iiop.IIOPSSLSocketFactory.createServerSocket(IIOP
      SSLSocketFactory.java:289)
      at com.sun.corba.ee.impl.transport.SocketOrChannelAcceptorImpl.initiali
      ze(SocketOrChannelAcceptorImpl.java:175)
      at com.sun.corba.ee.impl.transport.CorbaTransportManagerImpl.getAccepto
      rs(CorbaTransportManagerImpl.java:236)
      at com.sun.corba.ee.impl.transport.CorbaTransportManagerImpl.addToIORTe
      mplate(CorbaTransportManagerImpl.java:253)
      at com.sun.corba.ee.spi.oa.ObjectAdapterBase.initializeTemplate(ObjectA
      dapterBase.java:127)
      at com.sun.corba.ee.impl.oa.toa.TOAImpl.<init>(TOAImpl.java:107)
      at com.sun.corba.ee.impl.oa.toa.TOAFactory.getTOA(TOAFactory.java:98)
      at com.sun.corba.ee.impl.orb.ORBImpl.connect(ORBImpl.java:1664)
      at com.sun.corba.ee.spi.presentation.rmi.StubAdapter.connect(StubAdapte
      r.java:212)
      at com.sun.corba.ee.impl.orb.ORBImpl.getIOR(ORBImpl.java:2115)
      at com.sun.corba.ee.impl.orb.ORBImpl.getFVDCodeBaseIOR(ORBImpl.java:987
      )
      at com.sun.enterprise.util.ORBManager.initORB(ORBManager.java:556)
      at com.sun.enterprise.util.ORBManager.getORB(ORBManager.java:278)
      at com.sun.enterprise.util.ORBManager.getORB(ORBManager.java:289)
      at com.sun.enterprise.server.ondemand.EjbServiceGroup.createORB(EjbServ
      iceGroup.java:511)
      at com.sun.enterprise.server.ondemand.EjbServiceGroup.startORB(EjbServi
      ceGroup.java:437)
      at com.sun.enterprise.server.ondemand.EjbServiceGroup._start(EjbService
      Group.java:156)
      at com.sun.enterprise.server.ondemand.EjbServiceGroup.start(EjbServiceG
      roup.java:143)
      at com.sun.enterprise.server.ondemand.ServiceGroup$1.run(ServiceGroup.j
      ava:193)
      at java.security.AccessController.doPrivileged(Native Method)
      at com.sun.enterprise.server.ondemand.ServiceGroup.startChildren(Servic
      eGroup.java:190)
      at com.sun.enterprise.server.ondemand.MainServiceGroup.start(MainServic
      eGroup.java:58)
      at com.sun.enterprise.server.ondemand.ServerEntryListenerImpl.notifyEnt
      ry(ServerEntryListenerImpl.java:85)
      at com.sun.enterprise.server.ondemand.entry.ServerEntryHelper.sendEvent
      (ServerEntryHelper.java:75)
      at com.sun.enterprise.server.ondemand.entry.ServerEntryHelper.generateS
      tartUpEntryContext(ServerEntryHelper.java:64)
      at com.sun.enterprise.server.ondemand.OnDemandServer.generateEntryConte
      xt(OnDemandServer.java:154)
      at com.sun.enterprise.server.ondemand.OnDemandServer.onStartup(OnDemand
      Server.java:133)
      at com.sun.enterprise.server.PEMain.run(PEMain.java:409)
      at com.sun.enterprise.server.PEMain.main(PEMain.java:336)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl
      .java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcce
      ssorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:592)
      at com.sun.enterprise.server.PELaunch.main(PELaunch.java:415)

      #]

      [#|2009-09-12T23:27:03.291-0700|SEVERE|sun-appserver2.1|javax.enterprise.resour
      ce.corba.ee.S1AS-ORB.rpc.transport|_ThreadID=10;_ThreadName=main;all interfaces
      ;33820;;_RequestID=9340f5b1-fc93-4e40-abeb-b528e180af5b;|"IOP00410216: (COMM_FA
      ILURE) Unable to create IIOP listener on the specified host/port: all interface
      s/33820"
      org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 216 completed: No
      at com.sun.corba.ee.impl.logging.ORBUtilSystemException.createListenerF
      ailed(ORBUtilSystemException.java:3187)
      at com.sun.corba.ee.impl.logging.ORBUtilSystemException.createListenerF
      ailed(ORBUtilSystemException.java:3207)
      at com.sun.corba.ee.impl.transport.SocketOrChannelAcceptorImpl.initiali
      ze(SocketOrChannelAcceptorImpl.java:182)
      at com.sun.corba.ee.impl.transport.CorbaTransportManagerImpl.getAccepto
      rs(CorbaTransportManagerImpl.java:236)
      at com.sun.corba.ee.impl.transport.CorbaTransportManagerImpl.addToIORTe
      mplate(CorbaTransportManagerImpl.java:253)
      at com.sun.corba.ee.spi.oa.ObjectAdapterBase.initializeTemplate(ObjectA
      dapterBase.java:127)
      at com.sun.corba.ee.impl.oa.toa.TOAImpl.<init>(TOAImpl.java:107)
      at com.sun.corba.ee.impl.oa.toa.TOAFactory.getTOA(TOAFactory.java:98)
      at com.sun.corba.ee.impl.orb.ORBImpl.connect(ORBImpl.java:1664)
      at com.sun.corba.ee.spi.presentation.rmi.StubAdapter.connect(StubAdapte
      r.java:212)
      at com.sun.corba.ee.impl.orb.ORBImpl.getIOR(ORBImpl.java:2115)
      at com.sun.corba.ee.impl.orb.ORBImpl.getFVDCodeBaseIOR(ORBImpl.java:987
      )
      at com.sun.enterprise.util.ORBManager.initORB(ORBManager.java:556)
      at com.sun.enterprise.util.ORBManager.getORB(ORBManager.java:278)
      at com.sun.enterprise.util.ORBManager.getORB(ORBManager.java:289)
      at com.sun.enterprise.server.ondemand.EjbServiceGroup.createORB(EjbServ
      iceGroup.java:511)
      at com.sun.enterprise.server.ondemand.EjbServiceGroup.startORB(EjbServi
      ceGroup.java:437)
      at com.sun.enterprise.server.ondemand.EjbServiceGroup._start(EjbService
      Group.java:156)
      at com.sun.enterprise.server.ondemand.EjbServiceGroup.start(EjbServiceG
      roup.java:143)
      at com.sun.enterprise.server.ondemand.ServiceGroup$1.run(ServiceGroup.j
      ava:193)
      at java.security.AccessController.doPrivileged(Native Method)
      at com.sun.enterprise.server.ondemand.ServiceGroup.startChildren(Servic
      eGroup.java:190)
      at com.sun.enterprise.server.ondemand.MainServiceGroup.start(MainServic
      eGroup.java:58)
      at com.sun.enterprise.server.ondemand.ServerEntryListenerImpl.notifyEnt
      ry(ServerEntryListenerImpl.java:85)
      at com.sun.enterprise.server.ondemand.entry.ServerEntryHelper.sendEvent
      (ServerEntryHelper.java:75)
      at com.sun.enterprise.server.ondemand.entry.ServerEntryHelper.generateS
      tartUpEntryContext(ServerEntryHelper.java:64)
      at com.sun.enterprise.server.ondemand.OnDemandServer.generateEntryConte
      xt(OnDemandServer.java:154)
      at com.sun.enterprise.server.ondemand.OnDemandServer.onStartup(OnDemand
      Server.java:133)
      at com.sun.enterprise.server.PEMain.run(PEMain.java:409)
      at com.sun.enterprise.server.PEMain.main(PEMain.java:336)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl
      .java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcce
      ssorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:592)
      at com.sun.enterprise.server.PELaunch.main(PELaunch.java:415)
      Caused by: java.net.BindException: Address already in use
      at java.net.PlainSocketImpl.socketBind(Native Method)
      at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
      at java.net.ServerSocket.bind(ServerSocket.java:319)
      at java.net.ServerSocket.<init>(ServerSocket.java:185)
      at javax.net.ssl.SSLServerSocket.<init>(SSLServerSocket.java:106)
      at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.<init>(SSLServerSoc
      ketImpl.java:106)
      at com.sun.net.ssl.internal.ssl.SSLServerSocketFactoryImpl.createServer
      Socket(SSLServerSocketFactoryImpl.java:71)
      at com.sun.enterprise.iiop.IIOPSSLSocketFactory.createSSLServerSocket(I
      IOPSSLSocketFactory.java:406)
      at com.sun.enterprise.iiop.IIOPSSLSocketFactory.createServerSocket(IIOP
      SSLSocketFactory.java:289)
      at com.sun.corba.ee.impl.transport.SocketOrChannelAcceptorImpl.initiali
      ze(SocketOrChannelAcceptorImpl.java:175)
      ... 32 more

      #]

        Activity

        Hide
        vkoniecz added a comment -

        Impact by this so interested in its resolution.

        Show
        vkoniecz added a comment - Impact by this so interested in its resolution.
        Hide
        Ed Bratt added a comment -

        Defer to 2.1.2. Will try to address this in the next update.

        Show
        Ed Bratt added a comment - Defer to 2.1.2. Will try to address this in the next update.
        Hide
        km added a comment -

        owner of v2 jmx connector.

        Show
        km added a comment - owner of v2 jmx connector.
        Hide
        Nazrul added a comment -

        -> Byron

        Show
        Nazrul added a comment - -> Byron
        Hide
        Byron Nevins added a comment -

        Port choosing complete for 3.1

        Show
        Byron Nevins added a comment - Port choosing complete for 3.1

          People

          • Assignee:
            Byron Nevins
            Reporter:
            meenap
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: