Issue Details (XML | Word | Printable)

Key: GLASSFISH-15424
Type: Bug Bug
Status: Reopened Reopened
Priority: Major Major
Assignee: Nigel Deakin
Reporter: varunrupela
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
glassfish

[BigApps] [STRESS] ~17 occurences of "EOFException" warnings coming from JMS

Created: 03/Jan/11 08:54 PM   Updated: 15/Feb/13 12:21 PM  Due: 18/Jan/11
Component/s: jms
Affects Version/s: 3.1_b34
Fix Version/s: 4.0.1

Time Tracking:
Not Specified

File Attachments: 1. Text File eof-issue.log (1.92 MB) 03/Jan/11 08:54 PM - varunrupela
2. File server.log-instance101-24x1-gf-b37 (30 kB) 13/Jan/11 01:56 AM - varunrupela

Issue Links:
Dependency
 

Tags: 3_1-exclude 3_1-release-note-added 3_1-release-notes 3_1_1-scrubbed 3_1_2-exclude 3_1_2-release-note-added 3_1_2-release-notes
Participants: easarina, Nazrul, Nigel Deakin, Rebecca Parks, Satish Kumar, Scott Fordin and varunrupela


 Description  « Hide

See parent issue 15423 for details of the BigApps run that causes this issue to appear in the server logs. A server log that shows the issue has been attached.



Satish Kumar added a comment - 04/Jan/11 01:05 AM

This is a bug in MQ. See http://java.net/jira/browse/MQ-72 for the corresponding MQ issue. Amy Kang is currently working on it and based on her feedback, a fix for this issue will be a part of the next MQ integration ...


varunrupela added a comment - 04/Jan/11 01:59 AM

Satish,

This is a different issue from the MQ-72 issue. The log is attached to the issue. We need to check on the cause for the log.


Satish Kumar added a comment - 04/Jan/11 02:47 AM

As I had mentioned in my earlier comment, I suspect this issue may be caused due to http://java.net/jira/browse/MQ-72.

Lowering the priority of this issue to Minor as discussed with Varun. We will need to have a re-look at this once MQ-72 is fixed and then decide if any further action is required here or if we can close this issue.


Satish Kumar added a comment - 05/Jan/11 11:09 PM

Bumping-up the priority to Major based on Nazrul's feedback.

We plan to leave this issue open until the fix for MQ 72 has been integrated and the stress tests have been run again and observe if the exceptions are reappearing in the new test run.


Nazrul added a comment - 10/Jan/11 05:48 PM

Please confirm if MQ-72 resolved this problem.


varunrupela added a comment - 12/Jan/11 09:56 PM

The issue continues to exist in build 37.


varunrupela added a comment - 13/Jan/11 01:56 AM

Added one of the instance's server log after a 24x1 run of the same scenario.


Satish Kumar added a comment - 13/Jan/11 07:36 AM

Assigning this issue to Nigel as per our discussion this evening...


Nigel Deakin added a comment - 14/Jan/11 08:40 AM

These messages

[#|2011-01-13T11:14:06.600+0530|WARNING|glassfish3.1|javax.jms|_ThreadID=29;_ThreadName=Thread-1;|[I500]: Caught JVM Exception: java.io.EOFException: Trying to read 72 bytes. Already read 0 bytes.|#]

are warning messages. No errors or other messages were seen and there are no reports of the application behaving unexpectedly other than these messages.

Other users have reported receiving these messages periodically in earlier versions of MQ. There is a discussion here:
http://markmail.org/message/hh2ejjuxp5mo6njp#query:+page:1+mid:rumu3mg7unqbgm25+state:results
to which MQ team members responded.

This warning message is logged by the MQ client thread that is reading messages from a socket connected to the broker. (Note that even though the broker is embedded, since the broker is clustered, the client uses TCP to connect to the broker). The exception suggests that the client had received a "zero-length packet" from the broker. This message is often seen when the broker has failed, though this is not the case here. In that email discussion the MQ technical lead speculates that it is "probably a side effect of destination limits or TTL" and suggests that if there are no other problems the messages can be ignored.

Note that after this exception has been logged the client will attempt to recover the connection and carry on. This appears to have been the case here since no further messages were logged.

This is just a warning message, and is logged as such. Arguably GlassFish should never log warnings, but to suppress it might cause useful information to be lost if there are other problems. So it is proposed to close this bug as "not being a bug".


varunrupela added a comment - 17/Jan/11 01:19 AM

Based on the analysis, the issue can be waived for GF 3.1. The issue should remain open as it appears in GF build 37 and as the MQ team will fix it in the long-term. It is useful to keep this issue open as opposed to creating a new one, since it contains quite some information around the bug.


Nigel Deakin added a comment - 17/Jan/11 01:50 AM

Adding 3_1-exclude 3_1-release-notes tags. Note that this issue now only concerns the EOFExceptions and no other messages.

The release note should mention that very occasionally WARNING messages "java.io.EOFException: Trying to read 72 bytes. Already read 0 bytes" may be observed in the server log. If no other messages or exceptions are logged at the same time in either the server or broker logs these messages may be ignored.


varunrupela added a comment - 18/Jan/11 01:41 AM

set the target release


easarina added a comment - 18/Jan/11 12:29 PM

Was used b38 01/14. richAccess + SSL stress test on Win 2008 machines. Saw multiple "java.io.EOFException: Trying to read 72 bytes. Already read 0 bytes" warnings in server.log files. Was used jdk 1.6.0_23, 64 bit.


Nigel Deakin added a comment - 19/Jan/11 02:11 AM

Updated summary to make it clear that it relates to warning messages, not exceptions.


Scott Fordin added a comment - 23/Mar/11 06:55 PM

Need more info to add issue to 3.1 Release Notes. Is a release note still necessary?


Nigel Deakin added a comment - 25/Mar/11 03:08 AM

Please add the following text (which I gave in my comment on 17 Jan) to the release note:

Issue 15424: Very occasionally WARNING messages "java.io.EOFException: Trying to read 72 bytes. Already read 0 bytes" may be observed in the server log. If no other messages or exceptions are logged at the same time in either the server or broker logs these messages may be ignored.

Nigel


Scott Fordin added a comment - 15/Apr/11 10:38 AM

Added issue to 3.1 Release Notes.


Nigel Deakin added a comment - 14/Dec/11 09:55 AM

Excluding from 3.1.2 for the same reason it was excluded from 3.1.1. It should continue to be mentioned in the release note.


Rebecca Parks added a comment - 24/Jan/12 09:24 PM

If it's in the 3.1.1 Release Notes, it's carried over to 3.1.2 unless it's fixed in 3.1.2. There's no need to flag it for 3.1.2.


Nigel Deakin added a comment - 15/Feb/13 12:21 PM

In accordance with the project triage guidelines this is not needed for 4.0 and so has been deferred until 4.0.1. Setting "fix version" accordingly.