[GLASSFISH-20712] Session Replication (Possibly CNF error in web container?) Created: 19/Jul/13  Updated: 23/Apr/15

Status: Open
Project: glassfish
Component/s: web_container
Affects Version/s: 3.1.2.2
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: alev50 Assignee: Rajiv Mordani
Resolution: Unresolved Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

1 node, 1 cluster, 2 instances


Attachments: Zip Archive logs_180713_1612.zip    
Tags: javaee_ri_target

 Description   

Here is the scenario I played :

1 - Start cluster
2 - Deploy web app
3 - Connect through load balancer -> instance 2
4 - Log on
5 - Stop instance 2
6 - Refresh browser -> instance 1
7 - Session lost + StreamCorruptedException
8 - Stop cluster

Logs are attached.

Thanks for help.

Regards,

Anthony

-----------------------------------------------------------------------------------------------------------------
Fialli Joe joe.fialli@oracle.com via shoal.java.net

Thanks for providing a full set of server logs just containing the test scenario that is failing for you.

I just want to make an overall statement that we are trying to track down class loading issues in the web container.
So you might want to post this issue to glassfish alias instead of the shoal alias.
So it is overkill to turn on the shoal logging to FINEST since that subsystem could not even be responsible
for corrupted stream (grizzly is the transport for messages between cluster members and the web container passes a
byte of serialized content into Shoal messaging system and when trying to reconstitute replicated session,
the byte array is taken out of shoal messaging subsystem and deserialized should be deserialized using web container class path.
While the failure is stating corrupted stream, the failure is always exactly the same type code of "00". If it truely was a corrupted stream,
we would see different values all the time. The best bet for Type Code 00 in stream is a class not being found.
The ClassNotFoundException is getting consumed and not reported.

Possible reason for the ClassNotFoundException is the following.

My observations when looking at the server log you sent yesterday was that it was incorrect
for there not to be a Web Container classloader context to deserialize the session context
(after all, the class is the app loaded in the container, the default ObjectInputStream is not going
to be able to deserialize that.)

So this failure is significant and needs interpretation by someone who works on web container. It happens quite a bit in the server logs that you sent in.
I am not aware why this would happen. The following might explain trying to deserialize replicated content in web container without using a web container
class path.

[#|2013-07-18T16:00:02.639+0200|FINEST|glassfish3.1.2|org.apache.catalina.loader.WebappLoader|_ThreadID=10;_ThreadName=Thread-2;ClassName=null;MethodName=null;|getClasspath
java.lang.NoSuchMethodException: com.sun.enterprise.v3.server.APIClassLoaderServiceImpl$APIClassLoader.getClasspath()
at java.lang.Class.getMethod(Class.java:1607)
at org.apache.catalina.loader.WebappLoader.getClasspath(WebappLoader.java:1196)
at org.apache.catalina.loader.WebappLoader.setClassPath(WebappLoader.java:1145)
at org.apache.catalina.loader.WebappLoader.start(WebappLoader.java:692)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:5298)
at com.sun.enterprise.web.WebModule.start(WebModule.java:498)
at org.apache.catalina.core.ContainerBase.startChildren(ContainerBase.java:1518)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1184)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:995)
at org.apache.catalina.core.ContainerBase.startChildren(ContainerBase.java:1518)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1184)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:529)
at org.apache.catalina.startup.Embedded.start(Embedded.java:942)
at com.sun.enterprise.web.WebContainer.postConstruct(WebContainer.java:604)
at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:131)
at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:91)
at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:82)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67)
at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:139)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:78)
at org.glassfish.internal.data.EngineInfo.getContainer(EngineInfo.java:93)
at com.sun.enterprise.v3.services.impl.WebContainerStarter.startWebContainer(WebContainerStarter.java:202)
at com.sun.enterprise.v3.services.impl.WebContainerStarter.postConstruct(WebContainerStarter.java:134)
at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:131)
at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:91)
at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:82)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67)
at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:139)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:78)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:253)
at com.sun.enterprise.v3.server.AppServerStartup.doStart(AppServerStartup.java:145)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:136)
at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79)
at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63)
at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishImpl.start(OSGiGlassFishImpl.java:69)
at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97)
at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55)

#]

*************

Here is time of first stream corrupted warning log message.
#|2013-07-18T16:06:35.588+0200|WARNING|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.ha.session.management|_ThreadID=75;_ThreadName=Thread-2;|Exception occurred in getSession

java.io.StreamCorruptedException: invalid type code: 00

Here is ClassNotFoundException that was probably related to that failure. (Note that this is a nested anonymous class. Typically difficult to serialize correctly.)
So the class below is probably one to look at.

[#|2013-07-18T16:06:35.584+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(com.transat.ga2010.service.BanqueServiceImpl$$EnhancerByCGLIB$$fedb0661)|#]

[#|2013-07-18T16:06:35.584+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| --> Passing on ClassNotFoundException|#]

Here are a large number classes that are being looked for just before this ClassNotFoundException log message.
[#|2013-07-18T16:06:34.749+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| --> Passing on ClassNotFoundException|#]

Note that the classes are mostly UI classes related to ajax, java faces.

[#|2013-07-18T16:06:37.493+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(com.transat.ga2010.web.messages)|#]
[#|2013-07-18T16:06:37.508+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(com.transat.ga2010.web.messages_fr)|#]
[#|2013-07-18T16:06:37.514+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(com.transat.ga2010.web.messages_fr_FR)|#]

[#|2013-07-18T16:06:38.123+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(org.ajax4jsf.xml.serializer.XMLEntities)|#]
[#|2013-07-18T16:06:38.129+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(org.ajax4jsf.xml.serializer.XMLEntities_fr)|#]

[#|2013-07-18T16:06:38.137+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(org.ajax4jsf.xml.serializer.XMLEntities_fr_FR)|#]

[#|2013-07-18T16:06:35.921+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(org.ajax4jsf.component.AjaxViewRootBeanInfo)|#]

[#|2013-07-18T16:06:35.925+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.UIViewRootBeanInfo)|#]

[#|2013-07-18T16:06:35.926+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.UIComponentBaseBeanInfo)|#]

[#|2013-07-18T16:06:36.012+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(org.ajax4jsf.messages_fr)|#]

[#|2013-07-18T16:06:36.019+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(org.ajax4jsf.messages_fr_FR)|#]

[#|2013-07-18T16:06:37.178+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(com.sun.facelets.compiler.UIInstructionsBeanInfo)|#]

[#|2013-07-18T16:06:37.183+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(com.sun.facelets.compiler.UILeafBeanInfo)|#]

[#|2013-07-18T16:06:37.190+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(login)|#]
[#|2013-07-18T16:06:37.198+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(login_fr_FR)|#]

[#|2013-07-18T16:06:37.205+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.html.HtmlFormBeanInfo)|#]
[#|2013-07-18T16:06:37.207+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.UIFormBeanInfo)|#]

[#|2013-07-18T16:06:37.253+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.html.HtmlMessagesBeanInfo)|#]
[#|2013-07-18T16:06:37.255+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.UIMessagesBeanInfo)|#]
[#|2013-07-18T16:06:37.265+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(org.richfaces.component.html.HtmlInputTextBeanInfo)|#]
[#|2013-07-18T16:06:37.273+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.UIInputBeanInfo)|#]

[#|2013-07-18T16:06:37.277+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.UIOutputBeanInfo)|#]
[#|2013-07-18T16:06:37.326+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(org.richfaces.component.html.HtmlInputSecretBeanInfo)|#]

[#|2013-07-18T16:06:37.328+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(org.richfaces.component.html.HtmlInputSecretBeanInfo)|#]

[#|2013-07-18T16:06:37.341+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.html.HtmlCommandButtonBeanInfo)|#]
[#|2013-07-18T16:06:37.344+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.UICommandBeanInfo)|#]
[#|2013-07-18T16:06:37.392+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(javax.faces.component.html.HtmlSelectBooleanCheckboxBeanInfo)|#]

[#|2013-07-18T16:06:37.195+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=76;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(login_fr)|#]

[#|2013-07-18T16:06:35.582+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=loadClass;|loadClass(com.transat.ga2010.service.BanqueServiceImpl$$EnhancerByCGLIB$$fedb0661)|#]

[#|2013-07-18T16:06:35.582+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=loadClass;| Delegating to classloader1 org.glassfish.internal.api.DelegatingClassLoader@7a8f9805|#]

[#|2013-07-18T16:06:35.584+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=loadClass;| Searching local repositories|#]

[#|2013-07-18T16:06:35.584+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClass(com.transat.ga2010.service.BanqueServiceImpl$$EnhancerByCGLIB$$fedb0661)|#]

[#|2013-07-18T16:06:35.584+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(com.transat.ga2010.service.BanqueServiceImpl$$EnhancerByCGLIB$$fedb0661)|#]

[#|2013-07-18T16:06:35.584+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| --> Passing on ClassNotFoundException|#]

[#|2013-07-18T16:06:35.582+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=loadClass;|loadClass(com.transat.ga2010.service.BanqueServiceImpl$$EnhancerByCGLIB$$fedb0661)|#]

[#|2013-07-18T16:06:35.582+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=loadClass;| Delegating to classloader1 org.glassfish.internal.api.DelegatingClassLoader@7a8f9805|#]

[#|2013-07-18T16:06:35.584+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=loadClass;| Searching local repositories|#]

[#|2013-07-18T16:06:35.584+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClass(com.transat.ga2010.service.BanqueServiceImpl$$EnhancerByCGLIB$$fedb0661)|#]

[#|2013-07-18T16:06:35.584+0200|FINER|glassfish3.1.2|javax.enterprise.system.container.web.org.glassfish.web.loader|_ThreadID=75;_ThreadName=Thread-2;ClassName=org.glassfish.web.loader.WebappClassLoader;MethodName=findClass;| findClassInternal(com.transat.ga2010.service.BanqueServiceImpl$$EnhancerByCGLIB$$fedb0661)|#]



 Comments   
Comment by Ed Bratt [ 19/Jul/13 ]

Posting for Reporter. Clear virus scan.

Comment by Sanjeeb Sahoo [ 22/Jul/13 ]

The following exception happens because WebappLoader is reflectively calling getClasspath method in all class loaders in the class loader chain and APIClassLoader does not have such a method. From what I see WebappLoader does not consider this to be an issue, as it logs the exception in FINEST level. So, I am assigning the issue to web container folks to figure out why session replication does not happen.

"[#|2013-07-18T16:00:02.639+0200|FINEST|glassfish3.1.2|org.apache.catalina.loader.WebappLoader|_ThreadID=10;_ThreadName=Thread-2;ClassName=null;MethodName=null;|getClasspath
java.lang.NoSuchMethodException: com.sun.enterprise.v3.server.APIClassLoaderServiceImpl$APIClassLoader.getClasspath()

Thanks,
Sahoo

Comment by Shing Wai Chan [ 06/Aug/13 ]

Can you attach a test case for this?

Comment by alev50 [ 12/Aug/13 ]

Not really, I deploy my production web app on a cluster, I stop the current instance and replication fails on a StreamCorruptedException. How could I write a simple test case for this ?

Thanks,

Anthony

Comment by Shing Wai Chan [ 12/Aug/13 ]

We have a unit test for session replication. It is working fine. So, we need more information to debug this. Can you try to construct a unit test case?

Comment by Shing Wai Chan [ 13/Aug/13 ]

Assign to Rajiv for session replication issue.





[GLASSFISH-6971] Check for <distributable/> when deploying with Availability enabled Created: 24/Dec/08  Updated: 24/Sep/12

Status: In Progress
Project: glassfish
Component/s: web_container
Affects Version/s: v2.1
Fix Version/s: not determined

Type: Improvement Priority: Critical
Reporter: Alexis MP Assignee: Rajiv Mordani
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issuezilla Id: 6,971

 Description   

I believe it doesn't make sense to deploy an application on a GlassFish cluster with availability enabled
(checkbox in web console or --availabilityenabled=true using asadmin) when the <distributable/> tag is
not present in the web.xml descriptor.

GlassFish should log the event as an explicit warning and provide feedback to the user upon deployment.



 Comments   
Comment by kumara [ 01/Sep/09 ]

Changing version from 9.1.1 to v2.1 to reflect new name/version.

Comment by Hong Zhang [ 23/Mar/11 ]

Assign to web team for initial evaluation to see if it's applicable to the current release and what we could do for it

Comment by Tom Mueller [ 06/Mar/12 ]

Bulk update to change fix version to "not determined" for all issues still open but with a fix version for a released version.





[GLASSFISH-18994] Impossible to use a CUSTOM session manager Created: 11/Aug/12  Updated: 15/Aug/12

Status: Open
Project: glassfish
Component/s: web_container
Affects Version/s: 3.1.2_b23
Fix Version/s: None

Type: Bug Priority: Major
Reporter: rickyepoderi Assignee: Rajiv Mordani
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux 3.2.0-3-amd64 x86_64 GNU/Linux (debian wheezy)


Attachments: Text File distributable.log     Text File non-distributable.log    

 Description   

I am trying to implement a couchbase session manager for glassfish (see https://github.com/rickyepoderi/couchbase-manager) and it is absolutely impossible to add it as a CUSTOM type (which seems to be the more reasonable way of integrating).

The following link http://docs.oracle.com/cd/E18930_01/html/821-2415/gkmhr.html explains how to include a new manager but it does not work. You have to follow one of the specified persistence types defined by this class: http://java.net/projects/glassfish/sources/svn/content/trunk/main/appserver/web/web-glue/src/main/java/com/sun/enterprise/web/session/PersistenceType.java. So, at first time, I decided to use "custom" type (I repeat that it seems the more reasonable way) but again it was impossible. If you set your PersistenceStrategyBuilder as "custom" the class http://java.net/projects/glassfish/sources/svn/content/trunk/main/appserver/web/web-glue/src/main/java/com/sun/enterprise/web/SessionManagerConfigurationHelper.java backs the application to "meemory" type (no matter the application was distributable or not).

I'm going to attach some logs with javax.enterprise.system.container.web.level=FINEST that shows that that class fails back to memory type (setting the application to <distributable/> and not setting).

Now I am re-using the "coherence-web" type but I think is not the proper way (besides it gives me another problems). Please it would be possible that the "custom" tag could just pass through this class with no interference (custom means custom, so let me do whatever I want).






[GLASSFISH-18694] sessionCreated of @WebListener class called on failover Created: 06/May/12  Updated: 08/May/12

Status: Open
Project: glassfish
Component/s: web_container
Affects Version/s: 3.1.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: mhankus Assignee: Rajiv Mordani
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux, jdk 7u4, 2 local linstances of GF 3.1.2



 Description   

In HA configuration, sessionCreated of @WebListener is called on every failover. Simple output to log files (shown below), informs container that new session was created (even it has the same id).

sessionCreated: 336cd4f4ee8ed1d714ba01330547
sessionCreated2: org.apache.catalina.session.StandardSessionFacade@7d22aadd
sessionCreated: 336cd4f4ee8ed1d714ba01330547
sessionCreated2: org.apache.catalina.session.StandardSessionFacade@4e9f32ec
sessionCreated: 336cd4f4ee8ed1d714ba01330547
sessionCreated2: org.apache.catalina.session.StandardSessionFacade@1180775e
sessionCreated: 336cd4f4ee8ed1d714ba01330547
sessionCreated2: org.apache.catalina.session.StandardSessionFacade@7331693f

Spec is not clear about it, but when we assume, that sessionCreated is supposed to be called on failover, sessionDestroyed should also be called on failover, but it is not.

My Listener looks like that:

@WebListener
public class DebugHttpSessionListener implements HttpSessionListener {

    private static final Log LOG = LogFactory.getLog(DebugHttpSessionListener.class);

    @Override
    public void sessionCreated(HttpSessionEvent se) {
        LOG.info("sessionCreated: " + se.getSession().getId());
        LOG.info("sessionCreated2: " + se.getSession());
        
    }

    @Override
    public void sessionDestroyed(HttpSessionEvent se) {
        LOG.info("sessionDestroyed" + se.getSession().getId());
    }
}





[GLASSFISH-15649] Session Invalidation for SSO apps results in error 500 for modified-attribute Created: 21/Jan/11  Updated: 27/Jan/11

Status: Open
Project: glassfish
Component/s: failover
Affects Version/s: 3.1_b38
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: gopaljorapur Assignee: Rajiv Mordani
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive com.sun.dft.glassfish.http.failover.singlesignon.SingleSignOnTest23.zip     Zip Archive com.sun.dft.glassfish.http.failover.singlesignon.SingleSignOnTest24.zip    

 Description   

There are couple of SSO tests failing with modified session attribute. These tests pass with session and modified session scope.

The scenario is like this.

1. Access app timebaseform. Sign in using SSO.
2. Access app form-nonha. Use SSO token for authentication.
3. Fail serving instance
4. Access app formauth. Use SSO token for authentication. SSO failover works as expected.
5 Access app ssoformath. Use SSO token for authentication.
6. Add some attributes to ssoformauth
7. Invalidate ssoformauth session. This results in a 500.

The log is attached. From the logs there are no obvious problems. Few things I noticed

Step 1 is served by instance101. Its seems to replicate the SSO session into 2 instances . instance103 & instance106. From instance106 its removed immediately. (Omitted some log entries for clarity)

[#|2011-01-18T20:38:22.870-0800|FINE|glassfish3.1|org.shoal.ha.cache.command.save|_ThreadID=36;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.store.ReplicatedDataStore;MethodName=put;|SSOStore: done save(92D4565BB0F627DADD45CCE6B5BE54A1) to instance106|#]

[#|2011-01-18T20:38:22.873-0800|FINE|glassfish3.1|org.shoal.ha.cache.command.save|_ThreadID=36;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.store.ReplicatedDataStore;MethodName=put;|SSOStore: done save(92D4565BB0F627DADD45CCE6B5BE54A1) to instance106|#]

[#|2011-01-18T20:38:22.986-0800|FINE|glassfish3.1|org.shoal.ha.cache.command.save|_ThreadID=42;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.commands.SaveCommand;MethodName=writeObject;|SSOStore sending save_command for key = 92D4565BB0F627DADD45CCE6B5BE54A1; version = -6; lastAccessedAt = 1295411902873; to instance106|#]

[#|2011-01-18T20:38:24.232-0800|FINE|glassfish3.1|org.shoal.ha.cache.command.save|_ThreadID=44;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.store.ReplicatedDataStore;MethodName=put;|SSOStore: done save(92D4565BB0F627DADD45CCE6B5BE54A1) to instance103|#]

[#|2011-01-18T20:38:24.342-0800|FINE|glassfish3.1|org.shoal.ha.cache.command.save|_ThreadID=45;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.commands.SaveCommand;MethodName=writeObject;|SSOStore sending save_command for key = 92D4565BB0F627DADD45CCE6B5BE54A1; version = -5; lastAccessedAt = 1295411904232; to instance103|#]

instance106 log :

[#|2011-01-18T20:38:19.638-0800|FINE|glassfish3.1|org.shoal.ha.cache.command|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.interceptor.CommandHandlerInterceptor;MethodName=onReceive;|SSOStore: Received SaveCommand:33(92D4565BB0F627DADD45CCE6B5BE54A1) from instance101|#]

[#|2011-01-18T20:38:19.638-0800|FINE|glassfish3.1|org.shoal.ha.cache.command.save|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.commands.SaveCommand;MethodName=execute;|SSOStoreSaveCommand:33 received save_command for key = 92D4565BB0F627DADD45CCE6B5BE54A1 from instance101|#]

[#|2011-01-18T20:38:21.037-0800|FINE|glassfish3.1|org.shoal.ha.cache.command|_ThreadID=42;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.interceptor.CommandHandlerInterceptor;MethodName=onReceive;|SSOStore: Received StaleCopyRemoveCommand:40(92D4565BB0F627DADD45CCE6B5BE54A1) from instance101|#]

[#|2011-01-18T20:38:21.037-0800|FINE|glassfish3.1|org.shoal.ha.cache.command.remove|_ThreadID=42;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.commands.StaleCopyRemoveCommand;MethodName=execute;|********************* REMOVED STALE REPLICA: 92D4565BB0F627DADD45CCE6B5BE54A1 ** SENT BY: instance101|#]

Another issue is trying to save after the store is destroyed. From instance104 logs :

[#|2011-01-18T20:38:44.115-0800|FINE|glassfish3.1|org.shoal.ha.cache.store|_ThreadID=25;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.ReplicatedBackingStore;MethodName=destroy;|** StoreName = /form-nonha is destroyed |#]

[#|2011-01-18T20:38:44.206-0800|FINE|glassfish3.1|org.shoal.ha.cache.stats|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.command.CommandManager;MethodName=handleMessage;|Received message# 3 from instance102|#]

[#|2011-01-18T20:38:44.206-0800|FINE|glassfish3.1|org.shoal.ha.cache.command|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.interceptor.CommandHandlerInterceptor;MethodName=onReceive;|/form-nonha: Received ReplicationFramePayloadCommand: contains 1 commands from instance102|#]

[#|2011-01-18T20:38:44.206-0800|FINE|glassfish3.1|org.shoal.ha.cache.command|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.interceptor.CommandHandlerInterceptor;MethodName=onReceive;|/form-nonha: Received SaveCommand:33(c90fec3181464b7d1bededdd100a) from instance102|#]

[#|2011-01-18T20:38:44.207-0800|FINE|glassfish3.1|org.shoal.ha.cache.command.save|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.commands.SaveCommand;MethodName=execute;|/form-nonhaSaveCommand:33 received save_command for key = c90fec3181464b7d1bededdd100a from instance102|#]

instance 106 :
[#|2011-01-18T20:38:44.080-0800|FINE|glassfish3.1|org.shoal.ha.cache.store|_ThreadID=25;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.ReplicatedBackingStore;MethodName=destroy;|** StoreName = /form-nonha is destroyed |#]

[#|2011-01-18T20:38:45.170-0800|FINE|glassfish3.1|org.shoal.ha.cache.store|_ThreadID=30;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.ReplicatedBackingStore;MethodName=destroy;|** StoreName = /timebasedform is destroyed |#]

[#|2011-01-18T20:38:45.873-0800|FINE|glassfish3.1|org.shoal.ha.cache.store|_ThreadID=35;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.ReplicatedBackingStore;MethodName=destroy;|** StoreName = /ssoformauth is destroyed |#]

[#|2011-01-18T20:38:46.609-0800|FINE|glassfish3.1|org.shoal.ha.cache.store|_ThreadID=27;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.ReplicatedBackingStore;MethodName=destroy;|** StoreName = /formauth is destroyed |#]

[#|2011-01-18T20:38:46.778-0800|FINE|glassfish3.1|org.shoal.ha.cache.stats|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.command.CommandManager;MethodName=handleMessage;|Received message# 2 from instance102|#]

[#|2011-01-18T20:38:46.778-0800|FINE|glassfish3.1|org.shoal.ha.cache.command|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.interceptor.CommandHandlerInterceptor;MethodName=onReceive;|/formauth: Received ReplicationFramePayloadCommand: contains 1 commands from instance102|#]

[#|2011-01-18T20:38:46.778-0800|FINE|glassfish3.1|org.shoal.ha.cache.command|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.interceptor.CommandHandlerInterceptor;MethodName=onReceive;|/formauth: Received SaveCommand:33(c91372e21a1000b5e5fb02d0a3eb) from instance102|#]

[#|2011-01-18T20:38:46.778-0800|FINE|glassfish3.1|org.shoal.ha.cache.command.save|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.adapter.store.commands.SaveCommand;MethodName=execute;|/formauthSaveCommand:33 received save_command for key = c91372e21a1000b5e5fb02d0a3eb from instance102|#]

[#|2011-01-18T20:38:46.779-0800|FINE|glassfish3.1|org.shoal.ha.cache.command.save|_ThreadID=29;_ThreadName=Thread-1;ClassName=org.shoal.ha.cache.impl.store.StoreableDataStoreEntryUpdater;MethodName=executeSave;|StoreableEntryUpdater received: key = c91372e21a1000b5e5fb02d0a3eb; entry.version0; cmd.version0|#]



 Comments   
Comment by sonymanuel [ 23/Jan/11 ]

To reproduce :

Run the following targets under gf-ha-qe/functional/http.

One time setup ;

$ant run-testng-test -Dtestclass=com.sun.dft.glassfish.http.failover.SetPersistenceScope -Dsuitename=http-modified-attribute -Dtest.properties="-DpersistenceScope=modified-attribute"

$ant -f build-singlesignon.xml init-sso

To run the test :

$ant -f build-singlesignon.xml test -Dtest.number=23
or
$ant -f build-singlesignon.xml test -Dtest.number=24

Comment by Mahesh Kannan [ 23/Jan/11 ]

I followed the exact steps that Sony mentioned in my setup (gf-ha-dev-sb6-14) and both the tests passed. Sony, it will be great if you could attach the logs (with ha logger set to FINE).

Comment by sonymanuel [ 24/Jan/11 ]

Attaching test logs.

Mahesh, can you try running the test a few times. This I think is an intermittent issue.

Comment by Mahesh Kannan [ 24/Jan/11 ]

I am unable to reproduce this on my setup even after 10 attempts.

Looking into Sony's log snippet it looks like the there are two save from 101, first one to 106 and then to 101. Thats the reason why there was a StaleCopyRemove command sent to 106.

I am transferring this to Shing Wai to find out why messages like

"** StoreName = /formauth is destroyed" are appearing before SaveCommands are issued.

Comment by Shing Wai Chan [ 24/Jan/11 ]

I do not see the issue in a three instance cluster setup in my machine.

Comment by Shing Wai Chan [ 27/Jan/11 ]

Per discussion with Sony, we will downgrade the bug. He still see this issue when running all the tests together in dft.
He don't think there is a SSO issue as the SSO failover is happening as expected.

Reassign to Rajiv for further investigation.





[GLASSFISH-17945] glassfish/nodes directories remain after uninstall ssh node Created: 09/Dec/11  Updated: 09/Dec/11

Status: Open
Project: glassfish
Component/s: admin
Affects Version/s: 3.1.2_b14
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: lidiam Assignee: Rajiv Mordani
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

ogs-3.1.2-b14-12_08_2011.zip, solaris sparc



 Description   

Created ssh node on a remote solaris sparc machine (DAS is also on solaris sparc). Created an instance and started it. Stopped instance and deleted it. Uninstalled the node in Admin Console: clicked on Delete and Uninstall node. After uninstall completed the following directories were still present on the remote machine:

/export/home/j2eetest/3.1.2/glassfish3/glassfish/nodes

The installation directory was 3.1.2/glassfish3, thus glassfish/nodes should have been removed by the uninstall. The above directory is empty, so it's a minor issue.



 Comments   
Comment by Tom Mueller [ 09/Dec/11 ]

Assigning to Rajiv, who I have as the owner for install-node. Rajiv, do you still own install-node and uninstall-node. If not, who does?





Generated at Sat Aug 29 11:08:51 UTC 2015 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.