Issue Details (XML | Word | Printable)

Key: GLASSFISH-18402
Type: Bug Bug
Status: Open Open
Priority: Major Major
Assignee: Harshad Vilekar
Reporter: andydr
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
glassfish

EJB lookup problem with webstart; multithreaded use of same client-side proxy?

Created: 23/Feb/12 05:00 PM   Updated: 28/Feb/12 04:45 PM
Component/s: orb
Affects Version/s: 3.1.2_b18, 3.1.2_b19, 3.1.2_b20, 3.1.2_b21, 3.1.2_b22, 3.1.2_b23
Fix Version/s: None

Time Tracking:
Not Specified

File Attachments: 1. Text File FastReuseOfInitialContextInJWS.txt (8 kB) 25/Feb/12 12:00 AM - Tim Quinn


Tags:
Participants: andydr, Harshad Vilekar, marina vatkina and Tim Quinn


 Description  « Hide

Our enterprise application contains an EJB module and the application client. The remote jndi lookup in the application client works fine, if the client stub runs with the appclient shell script. But when the application is executed via webstart, the beans won't be located properly due to the following error:

Caused by: javax.naming.NamingException: ejb ref resolution error for remote business interfacede.beans.einkauf.MyServiceBeanRemote [Root exception is java.lang.IllegalArgumentException: java.lang.ClassCastException@532c81b7]
	at com.sun.ejb.EJBUtils.lookupRemote30BusinessObject(EJBUtils.java:434)
	at com.sun.ejb.containers.RemoteBusinessObjectFactory.getObjectInstance(RemoteBusinessObjectFactory.java:75)
	at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:304)
	at com.sun.enterprise.naming.impl.SerialContext.getObjectInstance(SerialContext.java:556)
	at com.sun.enterprise.naming.impl.SerialContext.lookup(SerialContext.java:514)
	... 15 more
Caused by: java.lang.IllegalArgumentException: java.lang.ClassCastException@532c81b7
	at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at com.sun.ejb.EJBUtils.lookupRemote30BusinessObject(EJBUtils.java:422)
	... 19 more

Maybe a cache issue, but we have no clue what goes wrong here.



Tim Quinn added a comment - 23/Feb/12 05:30 PM

Can you attach your application to this issue so we can try to reproduce the problem from here?

Also, did you try clearing the Java Web Start cache on your client system and relaunching?

(Yes, this requires Java Web Start to download all the GlassFish system JARs again which can take a while as I'm sure you've experienced.)


andydr added a comment - 23/Feb/12 07:24 PM

The application is very complex with about 60 EJBs and requires a JDBC datastore. I can try to build a corresponding example.

And yes, I cleared the Web Start cache.


Tim Quinn added a comment - 23/Feb/12 08:52 PM

I am getting some help from the EJB team to direct where we need to look.

Also, what Java version are you running on the client?


andydr added a comment - 23/Feb/12 09:43 PM

The client runs with 1.6.0_29.

Just some details on the ejb lookup. The @Remote annotated interface is
retrieved with InitialContext#lookup(<interfaceName>.class.getName()) and not injected with @EJB.

I found out, that the problematic remote interfaces are looked up multiple times during startup (in the main-method).
When I introduce lazy initialization for these stateless session beans and reuse it, the problem is gone.

During runtime multiple lookups for other stateless session beans are no problem.


marina vatkina added a comment - 23/Feb/12 10:19 PM

How does your impl class look like? Does it have a business method called 'create'?


andydr added a comment - 23/Feb/12 10:35 PM

The SLSB looks like this

@Stateless
public class MyServiceBean implements MyServiceBeanRemote {
    @Override
    public MyService create() {
        MyService art = new MyService();
        em.persist(art);
        return art;
    }
}

So, yes it has a business method 'create'. Is something wrong with this?


marina vatkina added a comment - 23/Feb/12 10:39 PM

Does it make any difference if you rename 'create' to something else?


andydr added a comment - 23/Feb/12 11:17 PM - edited

Marina, the problem is NOT gone, when I rename the 'create' method.

One very last question: Quite often I get the following error:

org.omg.CORBA.COMM_FAILURE: WARNUNG: IOP00410037: Timeout while reading data in buffer manager vmcid: OMG minor code: 37 completed: No

The configuration TCP Read Timeout is set to 30secs! Is there another ORB configuration parameter available to solve
this issue?


andydr added a comment - 23/Feb/12 11:33 PM - edited

Just a reminder, the problem (ClassCastException during ejb lookup) happens only under Webstart. When the application stub is launched via the appclient script, it does not occur!

Apologize Marina, after a couple of webstart startups I feel, the timeout problem relates to the ClassCastException. Since sometimes the application starts up without any problems, but sometimes with read timeout exception and ClassCastExc.

A cross-check with appclient script showed the read timeout exception but no ClassCastExc.


marina vatkina added a comment - 24/Feb/12 12:11 AM

com.sun.ejb.EJBUtils.lookupRemote30BusinessObject(EJBUtils.java:422) calls a generated "create(String)" method. So my initial guess was that you also have such method in your remote interface, and the match happens correctly in ACC and not in JWS. But if renaming didn't help, it's not the cause of the problem.


Tim Quinn added a comment - 24/Feb/12 04:17 AM

andydr,

Is it the Java Web Start launch that sometimes works and sometimes does not?

Or does Java Web Start always fail and appclient launches sometimes work and sometimes not?

I long time ago I think I have seen cases where ORB failures of certain types can be reported as other types of errors.
I am not sure that is happening in this case but given the ORB errors you see sometimes perhaps this is a similar case.


andydr added a comment - 24/Feb/12 07:41 AM - edited

Tim, the application launch works in either case. But the ClassCastException during ejb lookup occurs in JWS.

BTW: The ClassCastExc happens even without the read timeout and is also unrelated to specific beans. I.e. with each application startup the ClassCastExc happens for a different bean.

java.lang.ClassCastException@4c1e6823]
	at com.sun.ejb.EJBUtils.lookupRemote30BusinessObject(EJBUtils.java:434)

What about JIRA#15775, can this class loader issue be linked to my problem?

Thanks, Andreas


marina vatkina added a comment - 24/Feb/12 07:25 PM

Does lookup of the EJBs work in JWS?


Harshad Vilekar added a comment - 24/Feb/12 08:15 PM

andydr,

Try increasing com.sun.corba.ee.transport.ORBTCPTimeouts.

Also, try increasing com.sun.corba.ee.transport.ORBTCPConnectTimeouts. Defaults are: 250:60000:100:5000 (max is 60 seconds).
"It controls how the ORB behaves on the client side when attempting to connect to an IOR (the wire rep of an EJB reference)."

Please see this blog for details about these properties:
https://blogs.oracle.com/ejcorba/entry/more_on_tcp_timeouts_in
https://blogs.oracle.com/ejcorba/entry/client_side_transport_timeouts_and


Tim Quinn added a comment - 24/Feb/12 09:33 PM

Andreas,

When I asked earlier about the Java Web Start launch and the appclient launch, I meant the successful run of the client not just the initial launch.

Sorry for the confusion.

You asked whether this could be related to 15775. I am not sure. In that case the last
exception in the stack trace is quite different and originates from a different place in the code.

If you want to specify the properties that Harshad mentioned you can pass them on the URL to launch the app client:

http://host:port/path?prop=A=1&prop=B=2

would be equivalent to

java -DA=1 -DB=2 ...

on a normal java command line.


Tim Quinn added a comment - 24/Feb/12 11:05 PM

Andreas,

I meant to ask about this earlier. You mentioned that if you look up the SLSB only once, lazily, (instead of multiple times)
and reuse the same reference then you do not get the errors.

Using multiple look-ups, which look-up causes the error? The first one? Second one? etc.

Also, you said that the look-ups occur in the main method. Can you attach the two versions of the main method to this issue:
the one that does multiple look-ups and the one that does a single lazy look-up with reuse? Or, of you'd prefer, you could e-mail
them to me directly (tjquinn at java dot net) and I would of course keep them confidential.

That might help us reproduce the problem here. Thanks.


Tim Quinn added a comment - 24/Feb/12 11:58 PM

I have a very simple client that I use for various manual tests. I modified it to create a new InitialContext and then
use it to look up and use a stateless session bean.

It gave me the buffer timeout warning Andreas reported the first time I ran it using Java Web Start, but that did not
cause any class cast errors. And the next 7 times I ran it I saw no such warning.

(I will attach my Java Web Start trace file.) I ran the same client several times using the appclient command and
never saw the warning, but clearly that problem is intermittent so perhaps I was just lucky - or unlucky.

By the way, I did not notice any delay at all during the run with the error.


Tim Quinn added a comment - 25/Feb/12 12:00 AM

Attached Java Web Start trace of app client launch with ORB warning...but it completed successfully.


andydr added a comment - 26/Feb/12 10:55 PM - edited

After a couple of code changes I assume the problem relates to
asynchronous ejb access. Our swing gui loads during startup
the data concurrently. Some EJBs are access from the main thread
and other EJBs via the awt-event queue.

I removed all new Thread(..) and SwingUtilitites.* parts and the startup
works as expected. Is concurrent access of ejbs not supported?

In a simple concurrent test I could reproduce the buffer timeout issue but
not the classcast problem:

for (int i = 0; i < 30; i++) {
   if (i % 2 == 0) {
     new Thread() {

       @Override
       public void run() {
         ... execute ejb
       }
     }.start();
   } else {
     ... execute ejb
   }
}

Tim Quinn added a comment - 27/Feb/12 12:01 AM

Thanks for doing the additional investigation and posting your results.

The EJB itself on the server side is (if I remember the spec correctly) is threadsafe,
but the layers on the client side (the ORB particularly) could be a different matter entirely.

Harshad will know more, but from your findings it seems that using the same client-side
proxy from multiple threads is triggering the problem.


Tim Quinn added a comment - 28/Feb/12 04:45 PM

based on the latest information from Andreas I have added to the title of this issue and am transferring it to the ORB component.

I am not sure whether this is a bug or whether the use of a single EJB proxy from multiple threads is unsupported, but Harshad is the right person to answer.