[GLASSFISH-18392] Multiple Failover of IIOP does not happen on dynamic cluster when one of the instances is removed Created: 22/Feb/12  Updated: 15/Feb/13

Status: Open
Project: glassfish
Component/s: rmi_iiop_failover
Affects Version/s: 3.1.2_b20
Fix Version/s: future release

Type: Bug Priority: Major
Reporter: mzh777 Assignee: Harshad Vilekar
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

OEL 5 + Oracle JDK


Attachments: Zip Archive instancedeleteMultipleFO.zip     File MultiEJBApp.ear    
Issue Links:
Dependency
depends on GLASSFISH-15804 Multiple failover not happening on OE... Resolved
Related
is related to GLASSFISH-15746 Multiple Failover of IIOP does not ha... Open

 Description   

We were investigating issue 15804 and by adding delays to the test operations (before and after start-cluster, killing instances, accessing business methods), the multiple-failover tests passing. But even with those delays, the multiple failover of IIOP still does not happen on dynamic cluster when one of the instances is removed. In those two tests, there are similarities (the same EJB application is used, multiple fail-over, same exception on client side). But there difference also in terms of testing steps. File this separate issue to track the new scenario.

Run test with appclient: appclient ... com.sun.appserver.ee.tests.client.ClientDynamicClusterRemoveInstance MultipleFO
Exception was throw on client side:
[testng] javax.ejb.EJBException: java.rmi.MarshalException: CORBA COMM_FAILURE 1330446344 Maybe; nested exception is:
[testng] org.omg.CORBA.COMM_FAILURE: FINE: IOP00410008: Connection abort vmcid: OMG minor code: 8 completed: Maybe
[testng] at com.sun.appserver.ee.tests.ejb.stateful._SFSB3Remote_Wrapper.getName(com/sun/appserver/ee/tests/ejb/stateful/_SFSB3Remote_Wrapper.java)
[testng] at com.sun.appserver.ee.tests.client.ClientDynamicClusterRemoveInstance.runTest2(Unknown Source)
[testng] at com.sun.appserver.ee.tests.client.ClientDynamicClusterRemoveInstance.runMultipleFOTest(Unknown Source)
[testng] at com.sun.appserver.ee.tests.client.ClientDynamicClusterRemoveInstance.main(Unknown Source)
[testng] Caused by: java.rmi.MarshalException: CORBA COMM_FAILURE 1330446344 Maybe; nested exception is:
[testng] org.omg.CORBA.COMM_FAILURE: FINE: IOP00410008: Connection abort vmcid: OMG minor code: 8 completed: Maybe
[testng] at com.sun.corba.ee.impl.javax.rmi.CORBA.Util.mapSystemException(Util.java:259)
[testng] at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.privateInvoke(StubInvocationHandlerImpl.java:213)
[testng] at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.invoke(StubInvocationHandlerImpl.java:152)
[testng] at com.sun.corba.ee.impl.presentation.rmi.codegen.CodegenStubBase.invoke(CodegenStubBase.java:227)
[testng] at com.sun.appserver.ee.tests.ejb.stateful._SFSB3Remote_Remote_DynamicStub.getName(com/sun/appserver/ee/tests/ejb/stateful/_SFSB3Remote_Remote_DynamicStub.java)
[testng] ... 4 more
[testng] Caused by: org.omg.CORBA.COMM_FAILURE: FINE: IOP00410008: Connection abort vmcid: OMG minor code: 8 completed: Maybe
[testng] at sun.reflect.GeneratedConstructorAccessor39.newInstance(Unknown Source)
[testng] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
[testng] at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
[testng] at com.sun.corba.ee.spi.orbutil.logex.corba.CorbaExtension.makeException(CorbaExtension.java:248)
[testng] at com.sun.corba.ee.spi.orbutil.logex.corba.CorbaExtension.makeException(CorbaExtension.java:95)
[testng] at com.sun.corba.ee.spi.orbutil.logex.WrapperGenerator.handleFullLogging(WrapperGenerator.java:387)
[testng] at com.sun.corba.ee.spi.orbutil.logex.WrapperGenerator.access$400(WrapperGenerator.java:107)
[testng] at com.sun.corba.ee.spi.orbutil.logex.WrapperGenerator$2.invoke(WrapperGenerator.java:511)
[testng] at com.sun.corba.ee.spi.orbutil.proxy.CompositeInvocationHandlerImpl.invoke(CompositeInvocationHandlerImpl.java:99)
[testng] at $Proxy32.connectionAbort(Unknown Source)
[testng] at com.sun.corba.ee.impl.transport.SocketOrChannelConnectionImpl.doOptimizedReadStrategy(SocketOrChannelConnectionImpl.java:1537)
[testng] at com.sun.corba.ee.impl.transport.SocketOrChannelConnectionImpl.doWork(SocketOrChannelConnectionImpl.java:1084)
[testng] at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497)
[testng] at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540)
[testng] Caused by: org.omg.CORBA.COMM_FAILURE: FINE: IOP00410030: Exception in a blocking read on connection SocketOrChannelConnectionImpl[ java.nio.channels.SocketChannel[connected local=/10.133.185.4:61556 remote=asqeopteron1.us.oracle.com/10.133.169.163:23703] ESTABLISHED true true] with a temporary selector vmcid: OMG minor code: 30 completed: No
[testng] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[testng] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
[testng] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
[testng] at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
[testng] at com.sun.corba.ee.spi.orbutil.logex.corba.CorbaExtension.makeException(CorbaExtension.java:248)
[testng] at com.sun.corba.ee.spi.orbutil.logex.corba.CorbaExtension.makeException(CorbaExtension.java:95)
[testng] at com.sun.corba.ee.spi.orbutil.logex.WrapperGenerator.handleFullLogging(WrapperGenerator.java:387)
[testng] at com.sun.corba.ee.spi.orbutil.logex.WrapperGenerator.access$400(WrapperGenerator.java:107)
[testng] at com.sun.corba.ee.spi.orbutil.logex.WrapperGenerator$2.invoke(WrapperGenerator.java:511)
[testng] at com.sun.corba.ee.spi.orbutil.proxy.CompositeInvocationHandlerImpl.invoke(CompositeInvocationHandlerImpl.java:99)
[testng] at $Proxy32.exceptionBlockingReadWithTemporarySelector(Unknown Source)
[testng] at com.sun.corba.ee.impl.transport.SocketOrChannelConnectionImpl.blockingRead(SocketOrChannelConnectionImpl.java:1604)
[testng] at com.sun.corba.ee.impl.transport.SocketOrChannelConnectionImpl.doOptimizedReadStrategy(SocketOrChannelConnectionImpl.java:1501)
[testng] ... 3 more
[testng] Caused by: java.io.IOException: Connection reset by peer
[testng] at sun.nio.ch.FileDispatcher.read0(Native Method)
[testng] at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
[testng] at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)
[testng] at sun.nio.ch.IOUtil.read(IOUtil.java:171)
[testng] at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:243)
[testng] at com.sun.corba.ee.impl.transport.SocketOrChannelConnectionImpl.blockingRead(SocketOrChannelConnectionImpl.java:1567)
[testng] ... 4 more

The steps to reproduce:

  1. Deploy MultiEJBApp.ear
  2. stop-cluster st-cluster
  3. start-instance instance110 and instance109
  4. access business methods
  5. Find and kill the instance with message [SFSB1Bean.getName]
  6. Delete the instance with message [SFSB1Bean.getName]
  7. start-cluster st-cluster
  8. access business methods
  9. Find and kill the instance with message [SFSB1Bean.getName] again.
  10. access more business methods
  11. assert the test pass or fail


 Comments   
Comment by mzh777 [ 22/Feb/12 ]

Test app and full test logs.





[GLASSFISH-16323] Fix FOLB related FindBugs issues in GlassFish repository Created: 06/Apr/11  Updated: 02/Jun/11

Status: In Progress
Project: glassfish
Component/s: rmi_iiop_failover, rmi_iiop_load_balancer
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Ken Cavanaugh Assignee: Harshad Vilekar
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

There are a number of FindBugs issues in code related to IIOP FOLB that need to be fixed:

kcavanaugh: common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java:490: IS2_INCONSISTENT_SYNC: Inconsistent synchronization of com.sun.enterprise.naming.impl.RoundRobinPolicy.resolvedEndpoints; locked 66% of time
kcavanaugh: common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java:273: IS2_INCONSISTENT_SYNC: Inconsistent synchronization of com.sun.enterprise.naming.impl.RoundRobinPolicy.resolvedEndpoints; locked 66% of time
kcavanaugh: common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java:253: IS2_INCONSISTENT_SYNC: Inconsistent synchronization of com.sun.enterprise.naming.impl.RoundRobinPolicy.resolvedEndpoints; locked 66% of time
kcavanaugh: common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java:421: UPM_UNCALLED_PRIVATE_METHOD: Private method com.sun.enterprise.naming.impl.RoundRobinPolicy.getAddressPortList(List) is never called
kcavanaugh: common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java:132: UPM_UNCALLED_PRIVATE_METHOD: Private method com.sun.enterprise.naming.impl.RoundRobinPolicy.infoLog(String, Object[]) is never called
kcavanaugh: common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/SerialContext.java:238: NP_NULL_ON_SOME_PATH: Possible null pointer dereference of ? in new com.sun.enterprise.naming.impl.SerialContext(String, Hashtable, Habitat)
kcavanaugh: common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/SerialInitContextFactory.java:292: DLS_DEAD_LOCAL_STORE: Dead store to provider in com.sun.enterprise.naming.impl.SerialInitContextFactory.getInitialContext(Hashtable)
kcavanaugh: common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/SerialInitContextFactory.java:283: ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD: Write to static field com.sun.enterprise.naming.impl.SerialInitContextFactory.giso from instance method com.sun.enterprise.naming.impl.SerialInitContextFactory.getInitialContext(Hashtable)
kcavanaugh: common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/SerialInitContextFactory.java:271: ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD: Write to static field com.sun.enterprise.naming.impl.SerialInitContextFactory.rrPolicy from instance method com.sun.enterprise.naming.impl.SerialInitContextFactory.getInitialContext(Hashtable)
kcavanaugh: common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/TransientContext.java:744: UPM_UNCALLED_PRIVATE_METHOD: Private method com.sun.enterprise.naming.impl.TransientContext.print(Map) is never called

kcavanaugh: ejb/ejb-container/src/main/java/com/sun/ejb/EJBUtils.java:508: DLS_DEAD_LOCAL_STORE: Dead store to generatedRemoteIntf in com.sun.ejb.EJBUtils.loadGeneratedRemoteBusinessClasses(ClassLoader, String)
kcavanaugh: ejb/ejb-container/src/main/java/com/sun/ejb/EJBUtils.java:520: DLS_DEAD_LOCAL_STORE: Dead store to generatedRemoteWrapper in com.sun.ejb.EJBUtils.loadGeneratedRemoteBusinessClasses(ClassLoader, String)

kcavanaugh: orb/orb-iiop/src/main/java/org/glassfish/enterprise/iiop/impl/PEORBConfigurator.java:153: ITA_INEFFICIENT_TO_ARRAY: Method org.glassfish.enterprise.iiop.impl.PEORBConfigurator.configure(DataCollector, ORB) uses Collection.toArray() with zero-length array argument
kcavanaugh: orb/orb-iiop/src/main/java/org/glassfish/enterprise/iiop/impl/POAProtocolMgr.java:302: BC_UNCONFIRMED_CAST: Unchecked/unconfirmed cast from java.rmi.Remote to org.omg.CORBA.Object in org.glassfish.enterprise.iiop.impl.POAProtocolMgr.isIdentical(Remote, Remote)
kcavanaugh: orb/orb-iiop/src/main/java/org/glassfish/enterprise/iiop/impl/POAProtocolMgr.java:303: BC_UNCONFIRMED_CAST: Unchecked/unconfirmed cast from java.rmi.Remote to org.omg.CORBA.Object in org.glassfish.enterprise.iiop.impl.POAProtocolMgr.isIdentical(Remote, Remote)
kcavanaugh: orb/orb-iiop/src/main/java/org/glassfish/enterprise/iiop/impl/POARemoteReferenceFactory.java:543: BC_UNCONFIRMED_CAST: Unchecked/unconfirmed cast from org.omg.PortableServer.Servant to javax.rmi.CORBA.Tie in org.glassfish.enterprise.iiop.impl.POARemoteReferenceFactory.postinvoke(byte[], POA, String, Object, Servant)

kcavanaugh: orb/orb-connector/src/main/java/org/glassfish/enterprise/iiop/api/GlassFishORBHelper.java:79: IS2_INCONSISTENT_SYNC: Inconsistent synchronization of org.glassfish.enterprise.iiop.api.GlassFishORBHelper.protocolManager; locked 57% of time
kcavanaugh: orb/orb-connector/src/main/java/org/glassfish/enterprise/iiop/api/GlassFishORBHelper.java:123: IS2_INCONSISTENT_SYNC: Inconsistent synchronization of org.glassfish.enterprise.iiop.api.GlassFishORBHelper.protocolManager; locked 57% of time
kcavanaugh: orb/orb-connector/src/main/java/org/glassfish/enterprise/iiop/api/GlassFishORBHelper.java:134: IS2_INCONSISTENT_SYNC: Inconsistent synchronization of org.glassfish.enterprise.iiop.api.GlassFishORBHelper.protocolManager; locked 57% of time
kcavanaugh: orb/orb-connector/src/main/java/org/glassfish/enterprise/iiop/api/GlassFishORBHelper.java:142: IS2_INCONSISTENT_SYNC: Inconsistent synchronization of org.glassfish.enterprise.iiop.api.GlassFishORBHelper.protocolManager; locked 57% of time

kcavanaugh: security/ejb.security/src/main/java/com/sun/enterprise/iiop/security/IIOPSSLUtilImpl.java:157: BC_UNCONFIRMED_CAST: Unchecked/unconfirmed cast from org.omg.PortableInterceptor.IORInfo to com.sun.corba.ee.spi.legacy.interceptor.IORInfoExt in com.sun.enterprise.iiop.security.IIOPSSLUtilImpl.createSSLTaggedComponent(IORInfo, Object)



 Comments   
Comment by Harshad Vilekar [ 20/May/11 ]

Updated:
trunk/v3/orb/orb-connector/src/main/java/org/glassfish/enterprise/iiop/api/GlassFishORBHelper.java

Fixed following issues:
#orb-connector-1 orb/orb-connector/src/main/java/org/glassfish/enterprise/iiop/api/GlassFishORBHelper.java:79:
IS2_INCONSISTENT_SYNC: Inconsistent synchronization of org.glassfish.enterprise.iiop.api.GlassFishORBHelper.protocolManager; locked 57% of time

#orb-connector-2 orb/orb-connector/src/main/java/org/glassfish/enterprise/iiop/api/GlassFishORBHelper.java:123:
IS2_INCONSISTENT_SYNC: Inconsistent synchronization of org.glassfish.enterprise.iiop.api.GlassFishORBHelper.protocolManager; locked 57% of time

#orb-connector-3 orb/orb-connector/src/main/java/org/glassfish/enterprise/iiop/api/GlassFishORBHelper.java:134:
IS2_INCONSISTENT_SYNC: Inconsistent synchronization of org.glassfish.enterprise.iiop.api.GlassFishORBHelper.protocolManager; locked 57% of time

#orb-connector-4 orb/orb-connector/src/main/java/org/glassfish/enterprise/iiop/api/GlassFishORBHelper.java:142:
IS2_INCONSISTENT_SYNC: Inconsistent synchronization of org.glassfish.enterprise.iiop.api.GlassFishORBHelper.protocolManager; locked 57% of time
-----------------------

Updated trunk/v3/orb/orb-iiop/src/main/java/org/glassfish/enterprise/iiop/impl/PEORBConfigurator.java
Fixed following issue:
#orb-iiop-1 orb/orb-iiop/src/main/java/org/glassfish/enterprise/iiop/impl/PEORBConfigurator.java:153:
ITA_INEFFICIENT_TO_ARRAY: Method org.glassfish.enterprise.iiop.impl.PEORBConfigurator.configure(DataCollector,
ORB) uses Collection.toArray() with zero-length array argument
-----------------------

Comment by Harshad Vilekar [ 02/Jun/11 ]

Updated:
common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java

Fixed Following Issues:

1.1 common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java:490: IS2_INCONSISTENT_SYNC: Inconsistent synchronization of com.sun.enterprise.naming.impl.RoundRobinPolicy.resolvedEndpoints; locked 66% of time

1.2 common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java:273: IS2_INCONSISTENT_SYNC: Inconsistent synchronization of com.sun.enterprise.naming.impl.RoundRobinPolicy.resolvedEndpoints; locked 66% of time

1.3 common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java:253: IS2_INCONSISTENT_SYNC: Inconsistent synchronization of com.sun.enterprise.naming.impl.RoundRobinPolicy.resolvedEndpoints; locked 66% of time

2 common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java:421: UPM_UNCALLED_PRIVATE_METHOD: Private method com.sun.enterprise.naming.impl.RoundRobinPolicy.getAddressPortList(List) is never called

3 common/glassfish-naming/src/main/java/com/sun/enterprise/naming/impl/RoundRobinPolicy.java:132: UPM_UNCALLED_PRIVATE_METHOD: Private method com.sun.enterprise.naming.impl.RoundRobinPolicy.infoLog(String, Object[]) is never called

Test Status:

  • Default build tests: PASS
  • web-ql, full-ql and ejb dev tests: PASS




[GLASSFISH-16481] Dynamically-assigned IP addresses for cloud instances require changes to app client bootstrapping for remote RMI-IIOP access Created: 27/Apr/11  Updated: 08/Feb/12

Status: Open
Project: glassfish
Component/s: orb, rmi_iiop_failover, rmi_iiop_load_balancer
Affects Version/s: 4.0
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: Ken Cavanaugh Assignee: Harshad Vilekar
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

GF 3.2 will support clouds with both statically and dynamically (DHCP) assigned IP addresses. This has
an impact on supporting app clients that access EJBs remotely and to IIOP FOLB.

First, a couple of points about the ORB:

  • The ORB normally caches and re-uses TCP/IP connections aggressively, and holds TCP connections open for a long time.
  • Stateful EJBs have their current state on a single instance, and for efficiency we normally want each request to go to the same instance.
    o When a failure occurs, IIOP failover causes all EJBs that were accessed from the failed instance to failover to the same instance ("Sticky failover").

Typically a cloud-based cluster will have a load balancer at a well-known address for handling HTTP traffic.
The instances in the cloud may or may not be accessible over the public internet (if not, the load balancer sits
in some kind of DMZ so at least an address on the LB is publicly accessible). We have the following possible scenarios:

1. Only the LB is publicly accessible.
2. LB and instances are publicly accessible, static cluster.
3. LB and instances are publicly accessible, dynamic cluster.

Case 1 is discussed in RFE 16480.

Case 2 is probably no different than supporting IIOP FOLB in GF 3.1, so it needs no further discussion.

Case 3 is interesting, because it creates a bootstrapping problem. Currently IIOP FOLB assumes that there are
at least 2 fixed addresses of instances in the cluster which can be used to bootstrap into full FOLB operation.
This is not the case for the dynamic cluster, where an app client may find that all of the instance addresses have
been re-assigned every time it attempt to contact the cluster. The only available fixed, known address in this case
is that of the load balancer (which itself may will use DNS tricks to make more than one IP address available, both
for load balancing and high availability). So IIOP FOLB must make use of the LB address in this case.

While perhaps a suitable LB might be persuaded to support some sort of request that gives the client ORB the
information that it needs, I think Tim's suggestion of a small web listener is a better idea. Here something
in each GF 3.2 instance is listing to a URL like

http(s)://host:8080/__ORBS

which would respond to a GET request with a list of instance addresses in the cluster (and perhaps other information?).
This might be formatted using JSON for convenient processing.

It's not clear whether security is required here. If it is, it would add significant complexity and require integration with
the Java SE security facilities for managing a password or certificate.

I'm not sure how best to implement this in GF; I'm sure someone familiar with the web programming side of things could
point out how to implement this rather easily.

These issues are also summarized at the Dynamic IP address concerns page.






[GLASSFISH-16480] App client using RMI-IIOP needs to work in cloud without public IP addresses Created: 27/Apr/11  Updated: 08/Feb/12

Status: Open
Project: glassfish
Component/s: orb, rmi_iiop_failover, rmi_iiop_load_balancer
Affects Version/s: 4.0
Fix Version/s: future release

Type: New Feature Priority: Major
Reporter: Ken Cavanaugh Assignee: Harshad Vilekar
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

GF 3.2 will support clouds with both statically and dynamically (DHCP) assigned IP addresses. This has
an impact on supporting app clients that access EJBs remotely and to IIOP FOLB.

First, a couple of points about the ORB:

  • The ORB normally caches and re-uses TCP/IP connections aggressively, and holds TCP connections open for a long time.
  • Stateful EJBs have their current state on a single instance, and for efficiency we normally want each request to go to the same instance.
    o When a failure occurs, IIOP failover causes all EJBs that were accessed from the failed instance to failover to the same instance ("Sticky failover").

Typically a cloud-based cluster will have a load balancer at a well-known address for handling HTTP traffic.
The instances in the cloud may or may not be accessible over the public internet (if not, the load balancer sits
in some kind of DMZ so at least an address on the LB is publicly accessible). We have the following possible scenarios:

1. Only the LB is publicly accessible.
2. LB and instances are publicly accessible, static cluster.
3. LB and instances are publicly accessible, dynamic cluster.

In case 1, we have no choice but to direct IIOP traffic to the LB, assuming that the LB allows this.
If the application only uses stateless EJBs, we just run the ORB in a no-connection cache/connection per request mode
(we delivered this feature for NTT west around 5 years ago in GF 2.1.1). This allows the LB to make load balancing
decisions based on just a new TCP connection, and works reasonably well.

If the application requires stateful EJBs in case 1, we currently have no solution. I think a solution would require that the
LB somehow be aware of a sort of session affinity, where each request would be load balanced to the instance currently
containing the session for the EJB instance. A request that establishes a new session (currently that is the request
that a new InitialContext call sends) could be load balanced to any instance in the cluster. This would be very
similar to how the current IIOP LB mechanism works.

If the LB does not support IIOP, it would probably be necessary to use some sort of HTTP tunneling of IIOP requests.
This has been done many times in many products over the years (including in the circa 1995 Sun JOE ORB, probably the
first Java ORB written), but we have never supported it in the JDK/SJSAS/GF ORBs (which started in 1997). There is no
standard for tunneling IIOP over HTTP, but we could invent something proprietary to solve the problem if necessary.

Case 2 is probably no different than supporting IIOP FOLB in GF 3.1, so it needs no further discussion.

Case 3 creates a bootstrapping issue considered in another RFE.

These issues are also summarized at the Dynamic IP address concerns page.






[GLASSFISH-15656] IIOP failover sometime sends bad location forward to client Created: 21/Jan/11  Updated: 19/Sep/14

Status: Open
Project: glassfish
Component/s: rmi_iiop_failover
Affects Version/s: 3.1_b38
Fix Version/s: 4.1

Type: Bug Priority: Minor
Reporter: Ken Cavanaugh Assignee: Harshad Vilekar
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 3_1-exclude

 Description   

IIOP failover relies on getting notifications from the server when the cluster membership changes.
This is handled by sending a location forward to the client.
For unknown reasons, the client sometimes receives an incorrect location forward to
an object reference with a different type, causing subsequent calls to fail in the client
because the remote method does not exist. I patched this in 3.1 by rejecting such
illegal location forwards in the client, but I'd like to improve this a bit for 3.2:

1. Only reject illegal location forwards in the client that come from FOLB. The current
fix rejects these in CorbaContactInfoListImpl.setEffectiveTargetIOR. A better solution
would be to reject these in ClientGroupManager.receive_star, but that is a little more
difficult to implement.

2. Should find out why these are being sent and fix the underlying problem.



 Comments   
Comment by Tom Mueller [ 07/Feb/13 ]

Targeting for 4.0.1 since this fix is not needed for the Java EE 7 RI/SDK release.





Generated at Thu Aug 27 19:42:06 UTC 2015 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.