sailfin
  1. sailfin
  2. SAILFIN-1421

Conference, http connections: communication stopped, OOM, socket time out

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 1.0
    • Fix Version/s: milestone 1
    • Component/s: sip_container
    • Labels:
      None
    • Environment:

      Operating System: Linux
      Platform: All

      Description

      *****************************************************************************************************

      • Template v0.1 ( 05/01/08 )
      • Sailfin Stress test issue
        ******************************************************************************************************
        Sailfin Build :59
        Cluster size : 10
        Happens in a single instance (y/n) ? :
        Test id : st5_1_conference
        Location of the test : as-telco-sqe/stress-ws/conference
        JDK version : 1.6.0_07
        CLB used : Yes
        HW LB used : NO
        *******************************************
        Monitoring Enabled+ OLP enabled. I had such OLP setting:

      <property name="olpInserted" value="true"/>
      <property name="CpuOverloadRegulation" value="true"/>
      <property name="MmThreshold" value="100"/>
      <property name="SampleRate" value="5"/>
      <property name="SrThreshold" value="100"/>
      <property name="IrThreshold" value="100"/>

      *****************************************************************

      SuSE, one instance per the machine, one sipp per the instance.

      I've re-run this test two times.

      For both runs the communication stopped after few hours of the running. If to
      open conference-stats.txt for each instance, then will be seen that after a
      while less and less messages were sent and then nothing.

      From the beginning of the run in the outputs of http sessions I saw a lot of
      such WARNINGS:
      ======================================================================
      Nov 17, 2008 2:57:20 PM org.apache.commons.httpclient.HttpMethodBase getResponseBody
      [java] WARNING: Going to buffer response body of large or unknown size. Using
      getResponseBodyAsStream instead is recommended.
      =========================================================================

      Then after about 10-12 hours since the start of the test. Something happened
      for both runs.
      First run. For 2 http sessions were seen OOM errors.
      Second run. For 9 http sessions were seen OOM errors.

      For first run in outputs from 9 http sessions were seen
      SocketTimeoutExceptions. I did not see this exception during second run.

      I did not see errors in server.log files and sipp error logs.
      I've attached an output from one http session (second run).

      1. cn3.out
        5.19 MB
        easarina
      2. top.out
        129 kB
        easarina

        Issue Links

          Activity

          Hide
          varunrupela added a comment -

          Issue 1421 was not seen with the patch-fix provided for issue 1543.

          Show
          varunrupela added a comment - Issue 1421 was not seen with the patch-fix provided for issue 1543.
          Hide
          prasads added a comment -

          "Reclassifying as P4 because these issues are not must fix for SailFin 1.0 release.
          This issue will be scrubbed after this release and will be given the right
          priority for SailFin 2.0 release."

          Show
          prasads added a comment - "Reclassifying as P4 because these issues are not must fix for SailFin 1.0 release. This issue will be scrubbed after this release and will be given the right priority for SailFin 2.0 release."
          Hide
          easarina added a comment -

          SuSe machines, 10 instances cluster. Build 61 nightly 02/04/2009. I again saw
          this issue. The crash happened after about 12 hours in the running. At the final
          moment I saw OOM messages in http cliens outputs. The http machine had 16 GB of
          RAM.

          Show
          easarina added a comment - SuSe machines, 10 instances cluster. Build 61 nightly 02/04/2009. I again saw this issue. The crash happened after about 12 hours in the running. At the final moment I saw OOM messages in http cliens outputs. The http machine had 16 GB of RAM.
          Hide
          rampsarathy added a comment -

          A few perf enhancements has been done in the http path
          please re-run this test without SSR (latest 2.0 builds), if the issue is
          reproducible only with SSR , it should be re-assigned appropriately

          Show
          rampsarathy added a comment - A few perf enhancements has been done in the http path please re-run this test without SSR (latest 2.0 builds), if the issue is reproducible only with SSR , it should be re-assigned appropriately
          Hide
          easarina added a comment -

          With Sailfin 2.0 I don't see this problem any more, so this bug has to be closed.

          Show
          easarina added a comment - With Sailfin 2.0 I don't see this problem any more, so this bug has to be closed.

            People

            • Assignee:
              easarina
              Reporter:
              easarina
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: