saaj
  1. saaj
  2. SAAJ-31

OutOfMemory error occurs when sending large attachments

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: current
    • Fix Version/s: milestone 1
    • Component/s: www
    • Labels:
      None
    • Environment:

      Operating System: All
      Platform: All

    • Issuezilla Id:
      31

      Description

      We use SAAJ for processing SOAP requests containing MTOMized data. When
      receiving files of size +50MB we get the following exception. This problem is
      fixed by increasing the heap size. However the heap size would always have an
      upper limit on production systems and can not be increased beyond that limit.
      Is there a fix for this?

      java.lang.OutOfMemoryError: Java heap space
      at com.sun.xml.messaging.saaj.util.ByteOutputStream.ensureCapacity
      (ByteOutputStream.java:102)
      at com.sun.xml.messaging.saaj.util.ByteOutputStream.write
      (ByteOutputStream.java:109)
      at
      com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.find
      (BMMimeMultipart.java:562)
      at
      com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.readBody
      (BMMimeMultipart.java:338)
      at
      com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.getNextPart
      (BMMimeMultipart.java:274)
      at
      com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.parse
      (BMMimeMultipart.java:296)
      at
      com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.parse
      (BMMimeMultipart.java:212)
      at
      com.sun.xml.messaging.saaj.packaging.mime.internet.MimeMultipart.getCount
      (MimeMultipart.java:186)
      at com.sun.xml.messaging.saaj.soap.MessageImpl.initializeAllAttachments
      (MessageImpl.java:1257)
      at com.sun.xml.messaging.saaj.soap.MessageImpl.getAttachments
      (MessageImpl.java:759)
      at mytest.MTOMHandler.getSOAPAttachments(MTOMHandler.java:199)

        Activity

        Hide
        kumarjayanti added a comment -

        If you see our latest code,

        http://fisheye5.cenqua.com/browse/saaj/saaj-ri/src/java/com/sun/xml/messaging/saaj/soap/MessageImpl.java?r=1.9

        we no longer convert soap attachments to byte array before writing them to
        output stream. Only when there are no attachments we try to write from a
        messageBytes[] array to the stream.

        So if it is still failing with latest Jars then atleast the Stack-Trace cannot
        be the same as was reported earlier. So can you paste the stack-trace when you
        used the latest JARS, so that we trace where the issue is.

        Show
        kumarjayanti added a comment - If you see our latest code, http://fisheye5.cenqua.com/browse/saaj/saaj-ri/src/java/com/sun/xml/messaging/saaj/soap/MessageImpl.java?r=1.9 we no longer convert soap attachments to byte array before writing them to output stream. Only when there are no attachments we try to write from a messageBytes[] array to the stream. So if it is still failing with latest Jars then atleast the Stack-Trace cannot be the same as was reported earlier. So can you paste the stack-trace when you used the latest JARS, so that we trace where the issue is.
        Hide
        kumarjayanti added a comment -

        Can you please attach the stack-trace when using the latest builds from :

        https://saaj.dev.java.net/servlets/ProjectDocumentList?folderID=5118&expandFolder=5118&folderID=0

        Otherwise we think it is a duplicate of Issue 8 and would like to close the issue.

        Show
        kumarjayanti added a comment - Can you please attach the stack-trace when using the latest builds from : https://saaj.dev.java.net/servlets/ProjectDocumentList?folderID=5118&expandFolder=5118&folderID=0 Otherwise we think it is a duplicate of Issue 8 and would like to close the issue.
        Hide
        kashis added a comment -

        Here is the stack-trace with the new binaries.

        [With saaj.mime.optimization=false]

        java.lang.OutOfMemoryError: Java heap space
        com.sun.xml.messaging.saaj.util.ByteOutputStream.ensureCapacity(ByteOutputStream.java:133)
        com.sun.xml.messaging.saaj.util.ByteOutputStream.write(ByteOutputStream.java:122)
        com.sun.xml.messaging.saaj.packaging.mime.internet.MimeMultipart.parse(MimeMultipart.java:471)
        com.sun.xml.messaging.saaj.packaging.mime.internet.MimeMultipart.getBodyPart(MimeMultipart.java:249)
        com.sun.xml.messaging.saaj.soap.MessageImpl.init(MessageImpl.java:439)
        com.sun.xml.messaging.saaj.soap.MessageImpl.<init>(MessageImpl.java:309)
        com.sun.xml.messaging.saaj.soap.ver1_1.Message1_1Impl.<init>(Message1_1Impl.java:99)
        com.sun.xml.messaging.saaj.soap.ver1_1.SOAPMessageFactory1_1Impl.createMessage(SOAPMessageFactory1_1Impl.java:93)
        com.documentum.bps.webservice.soap.SOAPMessageSerDes.createSOAPMessageFromInputStream(SOAPMessageSerDes.java:52)
        com.documentum.bps.webservice.soap.SOAPMessageSerDes.readSOAPFromInputStream(SOAPMessageSerDes.java:81)
        com.documentum.bps.webservice.soap.SOAPMessageSerDes.readSOAPRequest(SOAPMessageSerDes.java:87)
        com.documentum.bps.webservice.inbound.runtime.WebServiceTaskCreator.createMessageContext(WebServiceTaskCreator.java:80)
        com.documentum.bps.webservice.inbound.runtime.WebServiceTaskCreator.createtTask(WebServiceTaskCreator.java:56)
        com.documentum.bps.inbound.WebServiceInboundServlet.doPost(WebServiceInboundServlet.java:72)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:803)

        [Without setting saaj.mime.optimization]

        java.lang.OutOfMemoryError: Java heap space
        com.sun.xml.messaging.saaj.util.ByteOutputStream.ensureCapacity(ByteOutputStream.java:133)
        com.sun.xml.messaging.saaj.util.ByteOutputStream.write(ByteOutputStream.java:140)
        com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.find(BMMimeMultipart.java:610)
        com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.readBody(BMMimeMultipart.java:368)
        com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.getNextPart(BMMimeMultipart.java:304)
        com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.parse(BMMimeMultipart.java:326)
        com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.parse(BMMimeMultipart.java:243)
        com.sun.xml.messaging.saaj.packaging.mime.internet.MimeMultipart.getCount(MimeMultipart.java:217)
        com.sun.xml.messaging.saaj.soap.MessageImpl.initializeAllAttachments(MessageImpl.java:1315)
        com.sun.xml.messaging.saaj.soap.MessageImpl.getAttachments(MessageImpl.java:795)
        com.documentum.bps.webservice.handlers.MTOMHandler.getSOAPAttachments(MTOMHandler.java:160)
        com.documentum.bps.webservice.handlers.MTOMHandler.handleRequest(MTOMHandler.java:69)
        com.documentum.bps.webservice.handlers.MTOMHandler.handleMessage(MTOMHandler.java:52)
        com.documentum.bps.webservice.handlers.HandlerProcessor.processChain(HandlerProcessor.java:18)
        com.documentum.bps.webservice.inbound.runtime.WebServiceTaskCreator.createtTask(WebServiceTaskCreator.java:62)
        com.documentum.bps.inbound.WebServiceInboundServlet.doPost(WebServiceInboundServlet.java:72)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:803)

        Show
        kashis added a comment - Here is the stack-trace with the new binaries. [With saaj.mime.optimization=false] java.lang.OutOfMemoryError: Java heap space com.sun.xml.messaging.saaj.util.ByteOutputStream.ensureCapacity(ByteOutputStream.java:133) com.sun.xml.messaging.saaj.util.ByteOutputStream.write(ByteOutputStream.java:122) com.sun.xml.messaging.saaj.packaging.mime.internet.MimeMultipart.parse(MimeMultipart.java:471) com.sun.xml.messaging.saaj.packaging.mime.internet.MimeMultipart.getBodyPart(MimeMultipart.java:249) com.sun.xml.messaging.saaj.soap.MessageImpl.init(MessageImpl.java:439) com.sun.xml.messaging.saaj.soap.MessageImpl.<init>(MessageImpl.java:309) com.sun.xml.messaging.saaj.soap.ver1_1.Message1_1Impl.<init>(Message1_1Impl.java:99) com.sun.xml.messaging.saaj.soap.ver1_1.SOAPMessageFactory1_1Impl.createMessage(SOAPMessageFactory1_1Impl.java:93) com.documentum.bps.webservice.soap.SOAPMessageSerDes.createSOAPMessageFromInputStream(SOAPMessageSerDes.java:52) com.documentum.bps.webservice.soap.SOAPMessageSerDes.readSOAPFromInputStream(SOAPMessageSerDes.java:81) com.documentum.bps.webservice.soap.SOAPMessageSerDes.readSOAPRequest(SOAPMessageSerDes.java:87) com.documentum.bps.webservice.inbound.runtime.WebServiceTaskCreator.createMessageContext(WebServiceTaskCreator.java:80) com.documentum.bps.webservice.inbound.runtime.WebServiceTaskCreator.createtTask(WebServiceTaskCreator.java:56) com.documentum.bps.inbound.WebServiceInboundServlet.doPost(WebServiceInboundServlet.java:72) javax.servlet.http.HttpServlet.service(HttpServlet.java:710) javax.servlet.http.HttpServlet.service(HttpServlet.java:803) [Without setting saaj.mime.optimization] java.lang.OutOfMemoryError: Java heap space com.sun.xml.messaging.saaj.util.ByteOutputStream.ensureCapacity(ByteOutputStream.java:133) com.sun.xml.messaging.saaj.util.ByteOutputStream.write(ByteOutputStream.java:140) com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.find(BMMimeMultipart.java:610) com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.readBody(BMMimeMultipart.java:368) com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.getNextPart(BMMimeMultipart.java:304) com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.parse(BMMimeMultipart.java:326) com.sun.xml.messaging.saaj.packaging.mime.internet.BMMimeMultipart.parse(BMMimeMultipart.java:243) com.sun.xml.messaging.saaj.packaging.mime.internet.MimeMultipart.getCount(MimeMultipart.java:217) com.sun.xml.messaging.saaj.soap.MessageImpl.initializeAllAttachments(MessageImpl.java:1315) com.sun.xml.messaging.saaj.soap.MessageImpl.getAttachments(MessageImpl.java:795) com.documentum.bps.webservice.handlers.MTOMHandler.getSOAPAttachments(MTOMHandler.java:160) com.documentum.bps.webservice.handlers.MTOMHandler.handleRequest(MTOMHandler.java:69) com.documentum.bps.webservice.handlers.MTOMHandler.handleMessage(MTOMHandler.java:52) com.documentum.bps.webservice.handlers.HandlerProcessor.processChain(HandlerProcessor.java:18) com.documentum.bps.webservice.inbound.runtime.WebServiceTaskCreator.createtTask(WebServiceTaskCreator.java:62) com.documentum.bps.inbound.WebServiceInboundServlet.doPost(WebServiceInboundServlet.java:72) javax.servlet.http.HttpServlet.service(HttpServlet.java:710) javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        Hide
        kumarjayanti added a comment -

        Hi,

        I would like to understand the need to make the getAttachments() call in your
        MTOMHandler ?. The moment you call getAttachments() the SAAJ API needs to parse
        the whole Mime package, detect the number of attachment parts and initialize
        Attachment Objects to be able to return the iterator.

        Also in the two stack traces, i believe the one where you did not set the [
        saaj.mime.optimization] flag to false the application fails at a later stage or
        does it fail at an earlier stage then in the case where you set the
        mime.optimization flag to "false" ?. Please confirm.

        Thanks.

        Show
        kumarjayanti added a comment - Hi, I would like to understand the need to make the getAttachments() call in your MTOMHandler ?. The moment you call getAttachments() the SAAJ API needs to parse the whole Mime package, detect the number of attachment parts and initialize Attachment Objects to be able to return the iterator. Also in the two stack traces, i believe the one where you did not set the [ saaj.mime.optimization] flag to false the application fails at a later stage or does it fail at an earlier stage then in the case where you set the mime.optimization flag to "false" ?. Please confirm. Thanks.
        Hide
        kumarjayanti added a comment -

        Fixed the issue. However we integrated MimePull parser into SAAJ to fix the
        issue. MimePull parser is a java.net project http://mimepull.dev.java.net/.

        The use of MimePull in SAAJ is not by default so one would need to set the
        following system property to allow the use :

        System.setProperty("saaj.use.mimepull", "true");

        In future once the system is fully tested we may remove the need for setting
        this property and SAAJ would use mimepull by default.

        Here is a test program with which i tested an 80MB attachment :

        public static void testSAAJIssue31() throws Exception

        { System.setProperty("saaj.use.mimepull", "true"); MessageFactory mf = MessageFactory.newInstance(); SOAPMessage m = mf.createMessage(); m.getSOAPBody().addTextNode("This is a test body"); SOAPHeader hdr = m.getSOAPHeader(); SOAPHeaderElement hdre = (SOAPHeaderElement)hdr.addChildElement("MYHeader","test", "http://tmpuri"); hdre.addTextNode("This is a test header"); m.saveChanges(); AttachmentPart ap = m.createAttachmentPart(new DataHandler(new FileDataSource("C:\\glassfish.zip"))); m.addAttachmentPart(ap); m.saveChanges(); m.writeTo(new FileOutputStream(new File("C:\\bigmessage.xml"))); SOAPMessage created = mf.createMessage(m.getMimeHeaders(), new FileInputStream(new File("C:\\bigmessage.xml"))); Iterator it = created.getAttachments(); AttachmentPart at = (AttachmentPart)it.next(); created.writeTo(new FileOutputStream(new File("C:\\bigmessage1.xml"))); }
        Show
        kumarjayanti added a comment - Fixed the issue. However we integrated MimePull parser into SAAJ to fix the issue. MimePull parser is a java.net project http://mimepull.dev.java.net/ . The use of MimePull in SAAJ is not by default so one would need to set the following system property to allow the use : System.setProperty("saaj.use.mimepull", "true"); In future once the system is fully tested we may remove the need for setting this property and SAAJ would use mimepull by default. Here is a test program with which i tested an 80MB attachment : public static void testSAAJIssue31() throws Exception { System.setProperty("saaj.use.mimepull", "true"); MessageFactory mf = MessageFactory.newInstance(); SOAPMessage m = mf.createMessage(); m.getSOAPBody().addTextNode("This is a test body"); SOAPHeader hdr = m.getSOAPHeader(); SOAPHeaderElement hdre = (SOAPHeaderElement)hdr.addChildElement("MYHeader","test", "http://tmpuri"); hdre.addTextNode("This is a test header"); m.saveChanges(); AttachmentPart ap = m.createAttachmentPart(new DataHandler(new FileDataSource("C:\\glassfish.zip"))); m.addAttachmentPart(ap); m.saveChanges(); m.writeTo(new FileOutputStream(new File("C:\\bigmessage.xml"))); SOAPMessage created = mf.createMessage(m.getMimeHeaders(), new FileInputStream(new File("C:\\bigmessage.xml"))); Iterator it = created.getAttachments(); AttachmentPart at = (AttachmentPart)it.next(); created.writeTo(new FileOutputStream(new File("C:\\bigmessage1.xml"))); }

          People

          • Assignee:
            saaj-issues
            Reporter:
            kashis
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: