array_repeat and array_insert raise inconsistent array size limit errors in codegen and interpreted execution

XMLWordPrintableJSON

      array_repeat and array_insert raise a different error for an array that exceeds the maximum array length depending on whether the expression runs through the interpreted path or through generated code.

      array_repeat

      eval    -> COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER   (SQLSTATE 54000)
      codegen -> _LEGACY_ERROR_TEMP_2176
      

      ArrayRepeat.eval checks count against ByteArrayMethods.MAX_ROUNDED_ARRAY_LENGTH and raises COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER. ArrayRepeat.genCodeForNumberOfElements performs no such check, so the generated code proceeds to the array allocation, which fails with the internal _LEGACY_ERROR_TEMP_2176. Codegen is the default execution path, so the internal error class is what users normally see. The interpreted path was migrated to the named condition by SPARK-45710 (4.0.0); the codegen path was not.

      array_insert

      eval    -> COLLECTION_SIZE_LIMIT_EXCEEDED.FUNCTION
      codegen -> COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER
      

      The codegen branch taken when pos is not foldable calls createArrayWithElementsExceedLimitError, which hardcodes "parameter" -> toSQLId("count"). The resulting message reads "the value of parameter(s) `count` in the function `array_insert` is invalid", but array_insert(array, pos, value) has no count parameter. Every other site in the expression – all three interpreted ones and the two sibling codegen ones – uses arrayFunctionWithElementsExceedLimitError (COLLECTION_SIZE_LIMIT_EXCEEDED.FUNCTION). Introduced by SPARK-56567 (4.2.0).

      Why this was not caught

      The existing test "Elements exceed limit for array_repeat()" in QueryExecutionErrorsSuite runs select array_repeat(1, 2147483647), which is constant-folded and therefore only exercises the interpreted path. The array_insert branch is reachable only with a non-foldable pos, because a foldable positive pos takes a different codegen branch.

      Scope

      Error reporting only – neither function produces wrong results, and the conditions under which each raises an error are unchanged.

      Sequence also reports COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER naming a count parameter it does not have, but it does so from sequenceLength, which both execution paths share. There is no interpreted-vs-codegen inconsistency there, so it is left alone.

            Assignee:
            sepuri sai krishna
            Reporter:
            sepuri sai krishna
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: