Issue Details (XML | Word | Printable)

Key: GRIZZLY-1254
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Unassigned
Reporter: lflince
Votes: 0
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
grizzly

This error occurs occasionally for no apparent reason, causing glassfish does not deliver all packets.

Created: 15/Apr/12 03:01 PM   Updated: 25/Apr/12 08:53 AM   Resolved: 25/Apr/12 08:53 AM
Component/s: ajp
Affects Version/s: 1.9.47
Fix Version/s: 1.9.49

Time Tracking:
Not Specified

File Attachments: 1. XML File domain.xml (30 kB) 15/Apr/12 06:38 PM - lflince
2. Text File glassfish-jk.properties.txt (0.2 kB) 15/Apr/12 06:38 PM - lflince
3. Java Archive File grizzly-http-ajp.jar (29 kB) 17/Apr/12 02:06 PM - oleksiys
4. Text File httpd.conf (19 kB) 15/Apr/12 08:04 PM - lflince
5. File log.7z (1.33 MB) 16/Apr/12 09:49 PM - lflince
6. Zip Archive log2.zip (61 kB) 17/Apr/12 10:59 AM - lflince
7. Text File server.log (1.63 MB) 16/Apr/12 09:08 PM - lflince

Environment:

Glassfish 3.1.2 build 23 Apache/2.2.9 mod_jk/1.2.32 Windows 2003 Server x64


Tags:
Participants: lflince and oleksiys


 Description  « Hide

[#|2012-04-14T20:36:21.593-0500|SEVERE|glassfish3.1.2|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=1333;_ThreadName=Thread-2;|GRIZZLY0051: ProcessorTask exception.
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkBounds(Buffer.java:559)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:143)
at com.sun.grizzly.util.InputReader.read(InputReader.java:216)
at com.sun.grizzly.http.ajp.AjpInputBuffer.ensureAvailable(AjpInputBuffer.java:210)
at com.sun.grizzly.http.ajp.AjpInputBuffer.readAjpMessageHeader(AjpInputBuffer.java:79)
at com.sun.grizzly.http.ajp.AjpProcessorTask.parseRequest(AjpProcessorTask.java:107)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:706)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1045)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:228)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
at java.lang.Thread.run(Thread.java:722)

#]

The log file is full of this error, several of these messages per second. The only way to stop it is going to: "server config \ network config \ protocols \ jk-connector \ HTTP modify" Max Connections value and SAVE. However, no matter how high the value set, this error will occur at some moment.



oleksiys added a comment - 15/Apr/12 05:12 PM

can you pls. try the patch from the issue
http://java.net/jira/browse/GLASSFISH-18446


lflince added a comment - 15/Apr/12 05:22 PM

When I had the 1.9.46 version, I tried that patch, but the error continued to appear. Now I have version 1.9.47, which I understand includes this patch.

http://java.net/jira/secure/ReleaseNote.jspa?projectId=10005&version=15532


oleksiys added a comment - 15/Apr/12 06:12 PM

can you pls. give more details on GF and apache config?
May be steps to reproduce the issue?


lflince added a comment - 15/Apr/12 06:38 PM

Attach more info, thanks.


oleksiys added a comment - 15/Apr/12 07:44 PM

can you pls. also attach apache config (httpd.conf)?
And if possible simple app we can use to reproduce the issue.


lflince added a comment - 15/Apr/12 08:04 PM

Attached httpd.conf
But a app not is simple


lflince added a comment - 16/Apr/12 01:48 AM

I do not know if this can help, from side Apache, the log file mod_jk.log, appears that, when errors occur:

[Sun Apr 15 19:55:06.843 2012] [3348:2152] [info] ajp_connection_tcp_get_message::jk_ajp_common.c (1266): (wk_glassfish1) can't receive the response header message from tomcat, tomcat (127.0.0.1:8009) has forced a connection close for socket 4616
[Sun Apr 15 19:55:06.843 2012] [3348:2152] [error] ajp_get_reply::jk_ajp_common.c (2118): (wk_glassfish1) Tomcat is down or refused connection. No response has been sent to the client (yet)


lflince added a comment - 16/Apr/12 04:30 AM

Sometimes glassfish can send this message:

Bad Gateway
The proxy server received an invalid response from an upstream server.


oleksiys added a comment - 16/Apr/12 09:17 AM

Can you pls. patch GF 3.1.2 (Grizzly version 1.9.47) w/ the attached jar (copy the attached jar into gfv3/glassfish/modules folder)?
It adds logging, so it would be easier to find the problem.

Thanks.


lflince added a comment - 16/Apr/12 12:12 PM

Thanks, I'll try


lflince added a comment - 16/Apr/12 09:08 PM

Attach new log file with error + other log info

Thanks


oleksiys added a comment - 16/Apr/12 09:28 PM

can you pls. attach all the log files in the folder (pls. compress them before)
thx.


oleksiys added a comment - 16/Apr/12 09:37 PM

the patch with more logs


lflince added a comment - 16/Apr/12 09:49 PM

Attach all log file after apply first patch


lflince added a comment - 16/Apr/12 09:52 PM

The new patch will have to apply when reduced traffic on the server


lflince added a comment - 17/Apr/12 10:59 AM

Attach new log files with error info

Thanks


oleksiys added a comment - 17/Apr/12 02:06 PM

this patch has to fix the issue.
can you pls. check it?

thx.


lflince added a comment - 17/Apr/12 02:19 PM

I'll try

Thanks


lflince added a comment - 19/Apr/12 04:35 PM

Yes, the issue is fixed, thanks oleksiys


oleksiys added a comment - 25/Apr/12 08:53 AM

fixed

Revision: c9e76d95f250590e2b670709079887027f3bddfa
Date: 2012-04-25 08:50:52 UTC

Log Message:
------------
+ fix issue #1254
http://java.net/jira/browse/GRIZZLY-1254
"This error occurs occasionally for no apparent reason, causing glassfish does not deliver all packets."