sailfin
  1. sailfin
  2. SAILFIN-2042

Heap usage per sip session (SUBSCRIBE) greatly increased.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Incomplete
    • Affects Version/s: 2.0
    • Fix Version/s: milestone 1
    • Component/s: sip_container
    • Labels:
      None
    • Environment:

      Operating System: Linux
      Platform: Sun

    • Issuezilla Id:
      2,042

      Description

      Heap usage per sip session (SUBSCRIBE) greatly increased in latest build.
      Test was done to create concurrently 400K sip dialogs per traffic instance,
      which was previously required ~ 2GB heap (~5kB per dialog), now taken >3GB, all
      traffic cluster instances got failed eventually and out of memory: Java heap
      space thrown in server.log for each traffic payload instance.

      Measured memory usage per session measured with early version Sun GlassFish
      Communications Server 2.0 ((v2.1 Patch06)(9.1_02 Patch12)) (build b31-fcs) was
      5.39545064 kB/ per User session.

      • JVM heap max set to (-Xms3000m )
      • SIP Session replication is disabled (default);
        --------------------------------------------------
        TibSim loader used to generated the following flow
        Cluster setup: 2 SC(oam) + 2 Traffic Instances

      SIP SUBSCRIBE refresh flow -----------------------

      Load gen SUT

      ------ SIP SUBSCRIBE (initial) ----->
      <----- 200 OK (subscribe) -----------
      <----- SIP NOTIFY -------------------
      ------ 200 OK (notify) ------------->
      ------ SIP SUBSCRIBE (refresh) ----->
      <----- 200 OK (subscribe) -----------

      Expire header set to 7200 (min)
      --------------------------------------------

      [#|2009-10-23T16:08:23.424-0400|SEVERE|sun-glassfish-comms-
      server2.0|javax.jms|_ThreadID=32;_ThreadName=iMQReadChannel-
      1;_RequestID=b1dc9b47-1dca-4b31-b008-d2f672c003a0;|java.lang.OutOfMemoryError:
      Java heap space
      java.lang.OutOfMemoryError: Java heap space
      at java.io.DataInputStream.<init>(DataInputStream.java:42)
      at com.sun.messaging.jmq.io.Packet.parseFixedBuffer(Packet.java:382)
      at com.sun.messaging.jmq.io.Packet.readPacket(Packet.java:1710)
      at com.sun.messaging.jmq.io.ReadOnlyPacket.readPacket
      (ReadOnlyPacket.java:109)
      at com.sun.messaging.jmq.io.ReadWritePacket.readPacket
      (ReadWritePacket.java:64)
      at
      com.sun.messaging.jmq.jmsclient.protocol.SocketConnectionHandler.readPacket
      (SocketConnectionHandler.java:39)
      at com.sun.messaging.jmq.jmsclient.ProtocolHandler.readPacket
      (ProtocolHandler.java:1762)
      at com.sun.messaging.jmq.jmsclient.ReadChannel.run
      (ReadChannel.java:1214)
      at java.lang.Thread.run(Thread.java:619)

      BUILD:
      Sun GlassFish Communications Server 2.0 ((v2.1 Patch06)(9.1_02 Patch12))
      (build b31d-fcs)

      1. DAS-server.zip
        161 kB
        tatyanalav
      2. PL-2-5-server.zip
        16 kB
        tatyanalav

        Activity

        Hide
        tatyanalav added a comment -

        Created an attachment (id=1171)
        Traffic Instance server log

        Show
        tatyanalav added a comment - Created an attachment (id=1171) Traffic Instance server log
        Hide
        tatyanalav added a comment -

        Created an attachment (id=1172)
        DAS server log ;

        Show
        tatyanalav added a comment - Created an attachment (id=1172) DAS server log ;
        Hide
        marcdumais added a comment -
            • Issue 2048 has been marked as a duplicate of this issue. ***
        Show
        marcdumais added a comment - Issue 2048 has been marked as a duplicate of this issue. ***
        Hide
        marcdumais added a comment -

        added myself in cc for this issue

        Show
        marcdumais added a comment - added myself in cc for this issue
        Hide
        prasads added a comment -

        Can you please let us know the following:

        1. When you measured the session size in b31-fcs , was it using a 32bit JDK or
        64 bit JDK ?
        2. Why are you using a Heap size of 2GB. With 64bit JDK , a higher heap size has
        been used and we have run the subscribe-refresh scenario successfully using 6GB
        Heap.
        Can you retry using a 6GB heap ?

        Show
        prasads added a comment - Can you please let us know the following: 1. When you measured the session size in b31-fcs , was it using a 32bit JDK or 64 bit JDK ? 2. Why are you using a Heap size of 2GB. With 64bit JDK , a higher heap size has been used and we have run the subscribe-refresh scenario successfully using 6GB Heap. Can you retry using a 6GB heap ?
        Hide
        Scott Oaks added a comment -

        I've did some measurements on builds 31 and 31g using the PresenceServlet we use
        for subscribe refresh testing, no session replication, and the sipp scenario
        described below.

        I took heap dumps of live objects (jmap -dump:live,file=filename) after creating
        8K sessions and used the Eclipse memory analyzer to examine the dumps. Here's
        what the measurement showed for object sizes:

        Total memory after loading 8K sessions: 69M in b31g; 80M in b31
        Size of 8K SipSessionDialogImpl: 272 in both builds
        Size of 8K HADialogFragment: 208 in both builds
        Size of 8K SipApplicationSessionImpl: 168 in both builds
        Size of 8K GeneralTimerImpl (session expiration): 104 in both builds

        The extra memory consumed in build 31 is due to lots of retained DOM-related
        objects all retained by serverbean config objects. I'm not sure why that's gone
        away in b31g, but at any rate, in our standard tests we are consuming less
        memory in b31g than in b31.

        So I'm not sure why the described test is running out of memory, but it doesn't
        appear to be due to the size of the default container session objects.

        Show
        Scott Oaks added a comment - I've did some measurements on builds 31 and 31g using the PresenceServlet we use for subscribe refresh testing, no session replication, and the sipp scenario described below. I took heap dumps of live objects (jmap -dump:live,file=filename) after creating 8K sessions and used the Eclipse memory analyzer to examine the dumps. Here's what the measurement showed for object sizes: Total memory after loading 8K sessions: 69M in b31g; 80M in b31 Size of 8K SipSessionDialogImpl: 272 in both builds Size of 8K HADialogFragment: 208 in both builds Size of 8K SipApplicationSessionImpl: 168 in both builds Size of 8K GeneralTimerImpl (session expiration): 104 in both builds The extra memory consumed in build 31 is due to lots of retained DOM-related objects all retained by serverbean config objects. I'm not sure why that's gone away in b31g, but at any rate, in our standard tests we are consuming less memory in b31g than in b31. So I'm not sure why the described test is running out of memory, but it doesn't appear to be due to the size of the default container session objects.
        Hide
        Scott Oaks added a comment -

        From Kevin Boone:

        Yes, further digging has revealed that they are, in fact, using the 64-bit JVM
        for the latest tests. So what they should really be saying is that the session
        size is larger with a 64-bit JVM, and not session session is larger with a later
        build (necessarily). The difference appears to be 5.4 kB per session as opposed
        to 3.2 kB per session.

        Show
        Scott Oaks added a comment - From Kevin Boone: Yes, further digging has revealed that they are, in fact, using the 64-bit JVM for the latest tests. So what they should really be saying is that the session size is larger with a 64-bit JVM, and not session session is larger with a later build (necessarily). The difference appears to be 5.4 kB per session as opposed to 3.2 kB per session.

          People

          • Assignee:
            sankara
            Reporter:
            tatyanalav
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: