After some days of debugging I seem to have discovered a critical file descriptor leak in all available jersey versions. The problem appears on the server side when multiple clients concurrently access any method of the defined resource and finally leads to leaked named pipes (i.e., FIFO pipes) which remain attached to the server process. As a consequence, the default system limit for open file descriptors (1024) is reached fast (depending on the amount of concurrent accesses) and parts of the service relying on streams (e.g., sockets) start failing. Note, that the problem does not appear if client access is delayed. For example, sending requests sequentially to the server every 5 seconds does not yield to leaked file descriptors.
The basic system setup to reproduce the bug looks as follows:
On the server side:
0. Maven with jersey 1.9 or any other version from the repository
1. Simple @Singleton resource with one POST method. For example:
public void testFdLeak(@FormParam("id") String test) ....
2. Lightweight HTTP server from JDK6 (HttpServer) to accept and forward the requests to the defined resource (same problem appears with Grizzly httpd as well)
On the client side:
Any client capable of sending a POST request can be used. For example we can execute the "curl" command concurrently with a basic bash script:
for i in $(seq 1 30);
curl -i -H "Content-Type: text/plain" -X POST -d "id=jersey" http://127.0.0.1/filedescriptor/testFdLeak
Result of the call on the server side:
When calling "lsof -p pid_of_the_server" or listing the file descriptors from the appropriate proc filesystem "ls -al /proc/PID/fd" directory and you will notice many named pipes starting and ending in the same process to remain open. For example:
java 13253 user 198r FIFO 0,8 0t0 267476 pipe
java 13253 user 199w FIFO 0,8 0t0 267476 pipe
java 13253 user 200u 0000 0,9 0 863 anon_inode
java 13253 user 201r FIFO 0,8 0t0 267477 pipe
java 13253 user 202w FIFO 0,8 0t0 267477 pipe
java 13253 user 203u 0000 0,9 0 863 anon_inode
Thank you very much in advance for a possible advice on how to work around this problem.