jaxp
  1. jaxp
  2. JAXP-38

Null pointer exception with decode calling substring_beforeF

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: current
    • Fix Version/s: milestone 1
    • Component/s: www
    • Labels:
      None
    • Environment:

      Operating System: All
      Platform: All

    • Issuezilla Id:
      38

      Description

      I have a fairly simple problem with a null pointer, that is easy to fix.
      This problem is a result of a different bug in the transform, that problem
      fails to set the value of a GLOBAL variable (rtf) correctly with a large style
      sheets file (it works ok with a small stylesheet), I have not isolated how to
      reproduce the original problem yet.
      But this is still a failure that is easy to resolve.

      In the file: http://fisheye5.cenqua.com/browse/jaxp-sources/xml-
      xalan/java/src/com/sun/org/apache/xalan/internal/xsltc/runtime/BasisLibrary.java
      ?r=1.10

      jeff suttor wrote:

      297 /**
      298 * XSLT Standard function substring-before().
      299 */
      300 public static String substring_beforeF(String value, String
      substring)

      { NEWPROPOSAL NEWPROPOSAL if (value == null || substring == null ) // Please add this NEWPROPOSAL return EMPTYSTRING; NEWPROPOSAL 301 final int index = value.indexOf(substring); 302 if (index >= 0) 303 return value.substring(0, index); 304 else 305 return EMPTYSTRING; 306 }

      Below is my trace back. I can send you the stylesheet file if you need it, but
      it is rather large and I didn't think it would help.

      javax.xml.transform.TransformerException: java.lang.NullPointerException
      at
      com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(Unknown
      Source)
      at
      com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(Unknown
      Source)
      at ibi.search.common.MagnifyTransform.<init>(MagnifyTransform.java:25)
      at ibi.search.lucene.LuceneCreateHTML.transformDOMToHTML
      (LuceneCreateHTML.java:704)
      at ibi.search.lucene.LuceneCreateHTML.transformToHTML
      (LuceneCreateHTML.java:629)
      at ibi.search.lucene.LuceneCreateHTML.<init>(LuceneCreateHTML.java:546)
      at ibi.search.lucene.LuceneSearch.<init>(LuceneSearch.java:422)
      at ibi.search.lucene.Lucene.doPost(Lucene.java:158)
      at ibi.search.lucene.Lucene.doGet(Lucene.java:142)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
      at ibi.srv.util.IBIHttpServlet.service(IBIHttpServlet.java:97)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
      at ibi.srv.util.IBIHttpServlet.service(IBIHttpServlet.java:106)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
      (ApplicationFilterChain.java:252)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter
      (ApplicationFilterChain.java:173)
      at org.apache.catalina.core.StandardWrapperValve.invoke
      (StandardWrapperValve.java:213)
      at org.apache.catalina.core.StandardContextValve.invoke
      (StandardContextValve.java:178)
      at org.apache.catalina.core.StandardHostValve.invoke
      (StandardHostValve.java:126)
      at org.apache.catalina.valves.ErrorReportValve.invoke
      (ErrorReportValve.java:105)
      at org.apache.catalina.core.StandardEngineValve.invoke
      (StandardEngineValve.java:107)
      at org.apache.catalina.connector.CoyoteAdapter.service
      (CoyoteAdapter.java:148)
      at org.apache.coyote.http11.Http11Processor.process
      (Http11Processor.java:869)
      at
      org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConne
      ction(Http11BaseProtocol.java:664)
      at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket
      (PoolTcpEndpoint.java:527)
      at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt
      (LeaderFollowerWorkerThread.java:80)
      at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run
      (ThreadPool.java:684)
      at java.lang.Thread.run(Unknown Source)
      Caused by: java.lang.NullPointerException
      at
      com.sun.org.apache.xalan.internal.xsltc.runtime.BasisLibrary.substring_beforeF
      (Unknown Source)
      at magnify_search_39.decode()
      at magnify_search_39.topLevel()
      at magnify_search_39.transform()
      at
      com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform
      (Unknown Source)

        Activity

        Hide
        Santiago Pericas-Geertsen added a comment -

        I don't understand the rationale for the proposal. Strings passed to that method are XSLT strings that
        should never be null. If they are, this is likely to be due to a bug elsewhere or the result of calling an
        external Java function that returns null. If there's another problem, we should focus on it first rather than
        trying to patch it. Please do attach a stylesheet to reproduce the problem. Setting priority to P3.

        Show
        Santiago Pericas-Geertsen added a comment - I don't understand the rationale for the proposal. Strings passed to that method are XSLT strings that should never be null. If they are, this is likely to be due to a bug elsewhere or the result of calling an external Java function that returns null. If there's another problem, we should focus on it first rather than trying to patch it. Please do attach a stylesheet to reproduce the problem. Setting priority to P3.
        Hide
        peter_lenahan added a comment -

        Well I managed to figure out my problem with a small style sheet.
        I am not sure if this is a bug on my part for referencing something
        in the wrong order. However, it did work in earlier versions.

        This was run against the most recent jaxp jar files.
        However it also occurs in the 1.6 version I am running.

        E:\test>java -cp ".;E:\test\jaxp-api.jar;e:\test\jaxp-ri.jar;" Transformer
        percent.xslt percent.xml percent.out

        decode: encoded=N%2FA

        ERROR: ''

        java.lang.NullPointerException

        The order that the variables are defined differs in the newer XSLT code.
        This order it seems didn't matter in the older version.

        It seems that the Global Variables with constants definitions were done
        before the variables set with template definitions in older version.
        Now this is not the case.

        Perhaps you should close the bug report for Issue 38.

        I am working with the Style Sheet from the Google Search Appliance,
        It works on the Google machine's XSLT transform.

        This is a run of the Transform done in Stylus Studio, it also works there.
        Here is the output in Stylus Studio:
        External XSLT processing started...
        file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl;
        Line #31; Column #19; decode: encoded=N%2FA
        file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl;
        Line #31; Column #19; decode: encoded=A
        file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl;
        Line #15; Column #15; value_declared_in_the_wrong_order=N/A
        file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl;
        Line #18; Column #15; FLD1=N%2FA&
        file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl;
        Line #31; Column #19; decode: encoded=FLD1=N%2FA&
        file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl;
        Line #31; Column #19; decode: encoded=A&
        file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl;
        Line #24; Column #15; Stuff=FLD1=N/A&
        ...done

        My "/" character is correctly decoded.

        <?xml version='1.0'?>
        <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
        <xsl:variable name="value_declared_in_the_wrong_order">
        <xsl:call-template name="decode">
        <xsl:with-param name="encoded" select="'N%2FA'"/>
        </xsl:call-template>
        </xsl:variable>

        <!--
        These variable are used by the decode template and are declared
        after the first call to the template above.
        -->
        <xsl:variable name="hex" select="'0123456789ABCDEF'"/>
        <xsl:variable name="ascii">
        !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz

        {|}

        ~</xsl:variable>
        <xsl:variable
        name="latin1"> ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ</xsl:variable>

        <xsl:template match="/">
        <xsl:message>value_declared_in_the_wrong_order=<xsl:value-of
        select="$value_declared_in_the_wrong_order"/></xsl:message>

        <xsl:message><xsl:value-of select="/percent"/></xsl:message>
        <xsl:variable name="stuff">
        <xsl:call-template name="decode">
        <xsl:with-param name="encoded" select="/percent"/>
        </xsl:call-template>
        </xsl:variable>
        <xsl:message>Stuff=<xsl:value-of select="$stuff"/></xsl:message>

        </xsl:template>

        <xsl:template name="decode">
        <xsl:param name="encoded"/>
        <xsl:message > decode: encoded=<xsl:value-of select="$encoded"/></xsl:message>
        <xsl:choose>
        <xsl:when test="contains($encoded,'%')">
        <xsl:value-of select="substring-before($encoded,'%')"/>
        <xsl:variable name="hexpair"
        select="translate(substring(substring-after($encoded,'%'),1,2),'abcdef','ABCDEF')"/>

        <!-- The variable $hex referenced below is undefined in the newer
        transform -->

        <xsl:variable name="decimal"
        select="(string-length(substring-before($hex,substring($hexpair,1,1))))*16 +
        string-length(substring-before($hex,substring($hexpair,2,1)))"/>

        <xsl:choose>
        <xsl:when test="$decimal < 127 and $decimal > 31">
        <xsl:value-of select="substring($ascii,$decimal - 31,1)"/>
        </xsl:when>
        <xsl:when test="$decimal > 159">
        <xsl:value-of select="substring($latin1,$decimal - 159,1)"/>
        </xsl:when>
        <xsl:otherwise>?</xsl:otherwise>
        </xsl:choose>

        <xsl:call-template name="decode">
        <xsl:with-param name="encoded"
        select="substring(substring-after($encoded,'%'),3)"/>
        </xsl:call-template>
        </xsl:when>
        <xsl:otherwise>
        <xsl:value-of select="$encoded"/>
        </xsl:otherwise>
        </xsl:choose>
        </xsl:template>
        </xsl:stylesheet>

        This is my XML FILE

        <?xml version="1.0"?>
        <percent>FLD1=N%2FA&</percent>

        Show
        peter_lenahan added a comment - Well I managed to figure out my problem with a small style sheet. I am not sure if this is a bug on my part for referencing something in the wrong order. However, it did work in earlier versions. This was run against the most recent jaxp jar files. However it also occurs in the 1.6 version I am running. E:\test>java -cp ".;E:\test\jaxp-api.jar;e:\test\jaxp-ri.jar;" Transformer percent.xslt percent.xml percent.out decode: encoded=N%2FA ERROR: '' java.lang.NullPointerException The order that the variables are defined differs in the newer XSLT code. This order it seems didn't matter in the older version. It seems that the Global Variables with constants definitions were done before the variables set with template definitions in older version. Now this is not the case. Perhaps you should close the bug report for Issue 38. I am working with the Style Sheet from the Google Search Appliance, It works on the Google machine's XSLT transform. This is a run of the Transform done in Stylus Studio, it also works there. Here is the output in Stylus Studio: External XSLT processing started... file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl ; Line #31; Column #19; decode: encoded=N%2FA file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl ; Line #31; Column #19; decode: encoded=A file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl ; Line #15; Column #15; value_declared_in_the_wrong_order=N/A file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl ; Line #18; Column #15; FLD1=N%2FA& file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl ; Line #31; Column #19; decode: encoded=FLD1=N%2FA& file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl ; Line #31; Column #19; decode: encoded=A& file:///d:/Documents%20and%20Settings/pl00291/workspace/lucene/WEB-INF/src/ibi/search/common/STY1A4.xsl ; Line #24; Column #15; Stuff=FLD1=N/A& ...done My "/" character is correctly decoded. <?xml version='1.0'?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:variable name="value_declared_in_the_wrong_order"> <xsl:call-template name="decode"> <xsl:with-param name="encoded" select="'N%2FA'"/> </xsl:call-template> </xsl:variable> <!-- These variable are used by the decode template and are declared after the first call to the template above. --> <xsl:variable name="hex" select="'0123456789ABCDEF'"/> <xsl:variable name="ascii"> !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz {|} ~</xsl:variable> <xsl:variable name="latin1"> ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ</xsl:variable> <xsl:template match="/"> <xsl:message>value_declared_in_the_wrong_order=<xsl:value-of select="$value_declared_in_the_wrong_order"/></xsl:message> <xsl:message><xsl:value-of select="/percent"/></xsl:message> <xsl:variable name="stuff"> <xsl:call-template name="decode"> <xsl:with-param name="encoded" select="/percent"/> </xsl:call-template> </xsl:variable> <xsl:message>Stuff=<xsl:value-of select="$stuff"/></xsl:message> </xsl:template> <xsl:template name="decode"> <xsl:param name="encoded"/> <xsl:message > decode: encoded=<xsl:value-of select="$encoded"/></xsl:message> <xsl:choose> <xsl:when test="contains($encoded,'%')"> <xsl:value-of select="substring-before($encoded,'%')"/> <xsl:variable name="hexpair" select="translate(substring(substring-after($encoded,'%'),1,2),'abcdef','ABCDEF')"/> <!-- The variable $hex referenced below is undefined in the newer transform --> <xsl:variable name="decimal" select="(string-length(substring-before($hex,substring($hexpair,1,1))))*16 + string-length(substring-before($hex,substring($hexpair,2,1)))"/> <xsl:choose> <xsl:when test="$decimal < 127 and $decimal > 31"> <xsl:value-of select="substring($ascii,$decimal - 31,1)"/> </xsl:when> <xsl:when test="$decimal > 159"> <xsl:value-of select="substring($latin1,$decimal - 159,1)"/> </xsl:when> <xsl:otherwise>?</xsl:otherwise> </xsl:choose> <xsl:call-template name="decode"> <xsl:with-param name="encoded" select="substring(substring-after($encoded,'%'),3)"/> </xsl:call-template> </xsl:when> <xsl:otherwise> <xsl:value-of select="$encoded"/> </xsl:otherwise> </xsl:choose> </xsl:template> </xsl:stylesheet> This is my XML FILE <?xml version="1.0"?> <percent>FLD1=N%2FA&</percent>
        Hide
        Santiago Pericas-Geertsen added a comment -

        Peter,

        Thanks for the detailed bug report and analysis. Indeed some code has changed
        in the way the processor initializes global variables (this is to allow certain
        dependencies with keys). The spec says that if variable x references variable y,
        the latter should be initialized first. In your stylesheet, however, this is
        much difficult to determine since you're calling another template. I suspect
        other processors will also fail in this case (but perhaps XSLTC fails more
        abrutly than others being a compiler).

        At least on the attached stylesheet, declaring all the variables before
        'value_declared_in_the_wrong_order' would solve the problem, right? In any case,
        it seems to me that changing the stylesheet would help your app run with
        different processors. I'm closing the bug, but feel free to re-open if you feel
        my analysis isn't correct.

        Show
        Santiago Pericas-Geertsen added a comment - Peter, Thanks for the detailed bug report and analysis. Indeed some code has changed in the way the processor initializes global variables (this is to allow certain dependencies with keys). The spec says that if variable x references variable y, the latter should be initialized first. In your stylesheet, however, this is much difficult to determine since you're calling another template. I suspect other processors will also fail in this case (but perhaps XSLTC fails more abrutly than others being a compiler). At least on the attached stylesheet, declaring all the variables before 'value_declared_in_the_wrong_order' would solve the problem, right? In any case, it seems to me that changing the stylesheet would help your app run with different processors. I'm closing the bug, but feel free to re-open if you feel my analysis isn't correct.
        Hide
        peter_lenahan added a comment -

        It is ok to close the bug, since it was my problem.

        But I wonder if a better error message scheme could somehow be generated.
        XSLT is often hard to debug and without a clue to where the problem happened it
        is really impossible. Just a null pointer exception message is a killer.

        Perhaps my un-initialized null variables could be initialized with an internal
        error message reference, so if the style sheet references one of these
        un-initialized variables, an error message string could be displayed instead of
        just crashing with a null pointer exception. I realized that this is a lot of
        work to add code to all the functions that work on the string variables, but
        XSLT is not exactly a friendly language without knowing where your problems are
        occurring. It would really help a lot to know where the problems originated.

        This may not be a perfect solution and perhaps the feature should also only be
        available when the setAttribute("debug",new Boolean(true)) is turned on.

        On the page: http://www.unicode.org/charts/PDF/UE000.pdf it defines private use
        UNICODE characters.

        Consider this possible feature.

        if (factory.getAttribute("debug"))

        { // // When you declare a variable initialize the variable // with a special character and reference to the // filename, line number // and character offset of the declaring variable. // Clearly this would need to be checked in hundreds of places // for the special first character. // char [] hidden=new char[1]; hidden[0]=0xE000; // This is a special use UNICODE Character. String special=hidden.toString()+ filenameReference+ linenumberReference+ characterOffsetReference; }

        Then when using the string if this value was set, it would tell the
        internal function that it was incorrect and where the problem
        originated.

        My problem was in a 6000 line style sheet which made the problem a little
        difficult to find and understand.

        Thanks,
        Peter

        Show
        peter_lenahan added a comment - It is ok to close the bug, since it was my problem. But I wonder if a better error message scheme could somehow be generated. XSLT is often hard to debug and without a clue to where the problem happened it is really impossible. Just a null pointer exception message is a killer. Perhaps my un-initialized null variables could be initialized with an internal error message reference, so if the style sheet references one of these un-initialized variables, an error message string could be displayed instead of just crashing with a null pointer exception. I realized that this is a lot of work to add code to all the functions that work on the string variables, but XSLT is not exactly a friendly language without knowing where your problems are occurring. It would really help a lot to know where the problems originated. This may not be a perfect solution and perhaps the feature should also only be available when the setAttribute("debug",new Boolean(true)) is turned on. On the page: http://www.unicode.org/charts/PDF/UE000.pdf it defines private use UNICODE characters. Consider this possible feature. if (factory.getAttribute("debug")) { // // When you declare a variable initialize the variable // with a special character and reference to the // filename, line number // and character offset of the declaring variable. // Clearly this would need to be checked in hundreds of places // for the special first character. // char [] hidden=new char[1]; hidden[0]=0xE000; // This is a special use UNICODE Character. String special=hidden.toString()+ filenameReference+ linenumberReference+ characterOffsetReference; } Then when using the string if this value was set, it would tell the internal function that it was incorrect and where the problem originated. My problem was in a 6000 line style sheet which made the problem a little difficult to find and understand. Thanks, Peter

          People

          • Assignee:
            Santiago Pericas-Geertsen
            Reporter:
            peter_lenahan
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: