jersey
  1. jersey
  2. JERSEY-2378

Memory leak when using HttpAuthenticationFeature (or HttpBasicAuthFilter)

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Cannot Reproduce
    • Affects Version/s: 2.4.1, 2.5, 2.5.1
    • Fix Version/s: 2.7
    • Component/s: core
    • Labels:
      None

      Description

      Hi,

      Jersey client seems to be massively leaking memory when HttpAuthenticationFeature (or HttpBasicAuthFilter in pre-2.5) is being applied repeatedly, like so:

      public static void main(String[] args) throws IOException {
      	System.out.println("Init");
      	Client client = ClientBuilder.newClient();
      	client.register(HttpAuthenticationFeature.basicBuilder().build());
      
      	System.in.read();
      	System.out.println("Start");
      	for (int i = 0; i < 1000; i++) {
      		WebTarget wt = client.target("http://non.exist.ent");
      		wt.property(HTTP_AUTHENTICATION_BASIC_USERNAME, "user");
      		wt.property(HTTP_AUTHENTICATION_BASIC_PASSWORD, "pass");
      		Entity ent = Entity.entity("Hello world!", MediaType.TEXT_PLAIN);
      		wt.request().async().post(ent);
      	}
      	System.out.println("Stop");
      	System.in.read();
      }
      

      The above code blows the heap up to around 200MB. If, however, username/password properties are not set, everything remains normal. The issue seems to be that a new lifecycle listener is added to the Client for each post() invocation when authentication is used, but only one gets added when there's no auth.

      Although the above example doesn't make much practical sense and this problem can in many cases be easily worked around by simply setting the credentials once (in this case outside of the loop), when a large number of different credentials are needed, this becomes a major problem, assuming that only a single, "everlasting" Client is used as the docs recommend.

        Activity

        Hide
        Miroslav Fuksa added a comment -

        Hi Andy,

        ok, thanks for the update.

        Answer to your question: If you close the response and don't keep a reference to the WebTarget then the memory should be released by GC. So, if the memory is still blocked, I think it is a bug. Comment from m.zdila probably verifies this assumption. We will check it.

        Mira

        Show
        Miroslav Fuksa added a comment - Hi Andy, ok, thanks for the update. Answer to your question: If you close the response and don't keep a reference to the WebTarget then the memory should be released by GC. So, if the memory is still blocked, I think it is a bug. Comment from m.zdila probably verifies this assumption. We will check it. Mira
        Hide
        m.zdila added a comment -

        Note that I am not sure if we are using endorsed Jersey 2.0 in Glassfisg 4.0 or 2.5.1 specified in maven dependencies of the application. If you need this to know I can check it at runtime.

        Show
        m.zdila added a comment - Note that I am not sure if we are using endorsed Jersey 2.0 in Glassfisg 4.0 or 2.5.1 specified in maven dependencies of the application. If you need this to know I can check it at runtime.
        Hide
        Miroslav Fuksa added a comment -

        yes, it would be good if you could find out the version. However, if you use plain GF 4.0 and you don't replace any jars inside you use probably 2.0.
        thanks. Mira

        Show
        Miroslav Fuksa added a comment - yes, it would be good if you could find out the version. However, if you use plain GF 4.0 and you don't replace any jars inside you use probably 2.0. thanks. Mira
        Hide
        Michal Gajdos added a comment -

        I am not able to reproduce the problem (cannot find any memory leak) in 2.7-SNAPSHOT. The first example (with setting properties directly into the request) and the second example work as expected. Feel free to reopen if you have other opinion.

        Show
        Michal Gajdos added a comment - I am not able to reproduce the problem (cannot find any memory leak) in 2.7-SNAPSHOT. The first example (with setting properties directly into the request) and the second example work as expected. Feel free to reopen if you have other opinion.
        Hide
        rumil86 added a comment -

        I have reproduced this problem in jersey-client 2.9.1. Code from first example results in OOM for me (increased the number of iterations to 10000).
        On the other hand setting client properties on Invocation.Builder instead of WebRequest does not lead to memory leak.
        So, to sum up, code that results in OOM (-Xmx128m, calling Response.close() doesn't change a thing):

         public static void main(String[] args) throws IOException {
                System.out.println("Init");
                Client client = ClientBuilder.newClient();
                client.register(HttpAuthenticationFeature.basicBuilder().build());
        
                System.out.println("Start");
                try {
                    for (int i = 0; i < 10000; i++) {
                        WebTarget wt = client.target("http://non.exist.ent");
                        wt.property(ClientProperties.READ_TIMEOUT, 1000);
                        wt.property(ClientProperties.CONNECT_TIMEOUT, 1000);
                        Entity ent = Entity.entity("Hello world!", MediaType.TEXT_PLAIN);
                        Invocation.Builder request = wt.request();
                        request.async().post(ent);
                    }
                } finally {
                    client.close();
                }
                System.out.println("Stop");
                System.in.read();
            }
        

        And code that works as expected:

         public static void main(String[] args) throws IOException {
                System.out.println("Init");
                Client client = ClientBuilder.newClient();
                client.register(HttpAuthenticationFeature.basicBuilder().build());
        
                System.out.println("Start");
                try {
                    for (int i = 0; i < 10000; i++) {
                        WebTarget wt = client.target("http://non.exist.ent");
                        Entity ent = Entity.entity("Hello world!", MediaType.TEXT_PLAIN);
                        Invocation.Builder request = wt.request();
                        request.property(ClientProperties.READ_TIMEOUT, 1000);
                        request.property(ClientProperties.CONNECT_TIMEOUT, 1000);
                        request.async().post(ent);
                    }
                } finally {
                    client.close();
                }
                System.out.println("Stop");
                System.in.read();
            }
        

        My investigation also confirms that there are too many JerseyClient.listeners, which are not needed but because a reference is held, they are not eligible for GC. These listeners are registered in org.glassfish.jersey.client.ClientConfig.State#initRuntime

        Show
        rumil86 added a comment - I have reproduced this problem in jersey-client 2.9.1. Code from first example results in OOM for me (increased the number of iterations to 10000). On the other hand setting client properties on Invocation.Builder instead of WebRequest does not lead to memory leak. So, to sum up, code that results in OOM (-Xmx128m, calling Response.close() doesn't change a thing): public static void main( String [] args) throws IOException { System .out.println( "Init" ); Client client = ClientBuilder.newClient(); client.register(HttpAuthenticationFeature.basicBuilder().build()); System .out.println( "Start" ); try { for ( int i = 0; i < 10000; i++) { WebTarget wt = client.target( "http: //non.exist.ent" ); wt.property(ClientProperties.READ_TIMEOUT, 1000); wt.property(ClientProperties.CONNECT_TIMEOUT, 1000); Entity ent = Entity.entity( "Hello world!" , MediaType.TEXT_PLAIN); Invocation.Builder request = wt.request(); request.async().post(ent); } } finally { client.close(); } System .out.println( "Stop" ); System .in.read(); } And code that works as expected: public static void main( String [] args) throws IOException { System .out.println( "Init" ); Client client = ClientBuilder.newClient(); client.register(HttpAuthenticationFeature.basicBuilder().build()); System .out.println( "Start" ); try { for ( int i = 0; i < 10000; i++) { WebTarget wt = client.target( "http: //non.exist.ent" ); Entity ent = Entity.entity( "Hello world!" , MediaType.TEXT_PLAIN); Invocation.Builder request = wt.request(); request.property(ClientProperties.READ_TIMEOUT, 1000); request.property(ClientProperties.CONNECT_TIMEOUT, 1000); request.async().post(ent); } } finally { client.close(); } System .out.println( "Stop" ); System .in.read(); } My investigation also confirms that there are too many JerseyClient.listeners, which are not needed but because a reference is held, they are not eligible for GC. These listeners are registered in org.glassfish.jersey.client.ClientConfig.State#initRuntime

          People

          • Assignee:
            Michal Gajdos
            Reporter:
            aketskes1
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 30 minutes
              30m
              Remaining:
              Remaining Estimate - 0 minutes
              0m
              Logged:
              Time Spent - 1 minute Time Not Required
              1m