Skip to content

fix(sql): fix internal error when INSERTing arrays represented as VARCHAR to DOUBLE[] columns#6060

Merged
ideoma merged 6 commits intomasterfrom
jh_internal_error_casting_vchar_to_array
Aug 20, 2025
Merged

fix(sql): fix internal error when INSERTing arrays represented as VARCHAR to DOUBLE[] columns#6060
ideoma merged 6 commits intomasterfrom
jh_internal_error_casting_vchar_to_array

Conversation

@jerrinot
Copy link
Copy Markdown
Contributor

Fixes an internal error that occurred when inserting VARCHAR values containing array representations into DOUBLE[] (or other array type) columns. The system now properly handles implicit casting from VARCHAR to array types during INSERT operations.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Aug 18, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Add VARCHAR (Utf8Sequence) support for writing STRING-like inputs into ARRAY columns by splitting array validation paths, adding a Utf8Sequence overload in RecordToRowCopierUtils, a corresponding implicitCastVarcharAsDoubleArray in SqlUtil, branching generated code to the VARCHAR path, and adding unit tests.

Changes

Cohort / File(s) Summary
Array transfer utils
core/src/main/java/io/questdb/griffin/RecordToRowCopierUtils.java, core/src/main/java/io/questdb/griffin/SqlUtil.java
Split the array-dimension validation/transfer into STRING and VARCHAR-specific paths; added validateArrayDimensionsAndTransferCol(..., Utf8Sequence vch, ...) overload and implicitCastVarcharAsDoubleArray(Utf8Sequence, DoubleArrayParser, int); updated generated branching to call the VARCHAR-specific path when source is VARCHAR; adjusted parser requirement logic.
Tests
core/src/test/java/io/questdb/test/griffin/InsertCastTest.java
Added testVarcharToArray() to verify successful INSERT of VARCHAR into DOUBLE[][], null handling, and rejection of invalid/emoji-containing inputs with appropriate inconvertible-value errors.

Sequence Diagram(s)

sequenceDiagram
  participant Copier as Generated copier
  participant Utils as RecordToRowCopierUtils
  participant Sql as SqlUtil
  participant Parser as DoubleArrayParser
  participant Row as TableWriter.Row

  Copier->>Utils: validateArrayDimensionsAndTransferCol(row, col, parser, vch, expectedType)
  Utils->>Sql: implicitCastVarcharAsDoubleArray(vch, parser, expectedType)
  Sql->>Parser: parse(vch.asAsciiCharSequence(), dims)
  Parser-->>Sql: ArrayView (or throw)
  Sql-->>Utils: ArrayView
  Utils->>Row: putArray(col, ArrayView)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch jh_internal_error_casting_vchar_to_array

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@jerrinot
Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Aug 18, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
core/src/test/java/io/questdb/test/griffin/InsertCastTest.java (2)

1342-1342: Typo: fix comment “bar array” → “bad array”

Minor nit.

Apply this diff:

-            // bar array
+            // bad array

1347-1351: Confirm intent: error message reports from-type as STRING even for VARCHAR input

The expected message uses [STRING -> DOUBLE[][]] although the input is cast as VARCHAR. This matches the current implementation (SqlUtil.implicitCastVarcharAsDoubleArray throws with from-type STRING). If this is the intended UX to unify error messaging across STRING/VARCHAR for array parsing, all good. If not, consider aligning the from-type to VARCHAR in both code and tests.

core/src/main/java/io/questdb/griffin/SqlUtil.java (1)

879-889: Clarify why exceptions use from-type STRING for VARCHAR input

The implementation intentionally throws ImplicitCastException with from-type STRING for VARCHAR inputs. This matches tests, but differs from other VARCHAR casts (which report VARCHAR). Consider documenting this to avoid confusion for future maintainers.

Apply this diff to add a brief rationale comment:

     public static ArrayView implicitCastVarcharAsDoubleArray(Utf8Sequence value, DoubleArrayParser parser, int expectedType) {
+        // Note: We deliberately report the source type as STRING in error messages, even though the input is VARCHAR.
+        // Arrays are parsed from an ASCII CharSequence (value.asAsciiCharSequence()), and existing array-cast errors
+        // are standardized to [STRING -> <ARRAY_TYPE>] across STRING/VARCHAR inputs.
         try {
             parser.of(value.asAsciiCharSequence(), ColumnType.decodeArrayDimensionality(expectedType));
         } catch (IllegalArgumentException e) {
             throw ImplicitCastException.inconvertibleValue(value, ColumnType.STRING, expectedType);
         }
         if (expectedType != ColumnType.UNDEFINED && parser.getType() != expectedType) {
             throw ImplicitCastException.inconvertibleValue(value, ColumnType.STRING, expectedType);
         }
         return parser;
     }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 281eea4 and 8e051f7.

📒 Files selected for processing (3)
  • core/src/main/java/io/questdb/griffin/RecordToRowCopierUtils.java (5 hunks)
  • core/src/main/java/io/questdb/griffin/SqlUtil.java (1 hunks)
  • core/src/test/java/io/questdb/test/griffin/InsertCastTest.java (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: New pull request (Check Changes Check changes)
  • GitHub Check: build
  • GitHub Check: build
🔇 Additional comments (7)
core/src/test/java/io/questdb/test/griffin/InsertCastTest.java (1)

1331-1352: Solid coverage for VARCHAR → DOUBLE[][] happy-path, null, and failure cases

  • The positive path and null handling look good.
  • The two negative cases are valuable: non-array text and non-ASCII content.
core/src/main/java/io/questdb/griffin/RecordToRowCopierUtils.java (6)

175-177: Pool both overloads for STRING/VARCHAR array transfers

Good addition. This allows the generator to emit the right helper call based on source column type.


275-277: Enable array parser path for both STRING and VARCHAR sources

Correctly broadens the special-case stack setup so the parser is available for VARCHAR→ARRAY casts too.


721-729: Correct bytecode sequence for VARCHAR → ARRAY transfer

  • Stack setup is consistent: [rowWriter, toColIdx, parser, record, fromColIdx] → getVarcharA → [rowWriter, toColIdx, parser, varchar] → push toColumnType → static helper.
  • Using the VARCHAR-specific helper ensures Utf8Sequence is handled without intermediate allocations beyond the ascii view.

Looks good.


817-818: Mirror STRING path to dedicated helper

Switching the STRING branch to the String-specific helper maintains symmetry and keeps the method signatures clean in the generator.


1188-1197: New VARCHAR helper: null-safe and delegates to SqlUtil

The null short-circuit preserves TableWriter/WAL null semantics. Delegation to SqlUtil centralizes parsing and error messaging. Good separation of concerns.


1203-1206: Array parser is now required for STRING and VARCHAR sources

Accurate capability check; avoids unnecessary parser field for unrelated copy plans.

@jerrinot
Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Aug 18, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@jerrinot
Copy link
Copy Markdown
Contributor Author

unrelated test failure, created #6062

mtopolnik
mtopolnik previously approved these changes Aug 18, 2025
@jerrinot
Copy link
Copy Markdown
Contributor Author

another failure due to #6062
this is annoying
Screenshot From 2025-08-19 09-31-22

@ideoma ideoma changed the title fix(sql): internal error when INSERTing arrays represented as VARCHAR to DOUBLE[] columns fix(sql): fix internal error when INSERTing arrays represented as VARCHAR to DOUBLE[] columns Aug 19, 2025
@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 18 / 19 (94.74%)

file detail

path covered line new line coverage
🔵 io/questdb/griffin/RecordToRowCopierUtils.java 13 14 92.86%
🔵 io/questdb/griffin/SqlUtil.java 5 5 100.00%

@ideoma ideoma merged commit 75335d3 into master Aug 20, 2025
35 checks passed
@ideoma ideoma deleted the jh_internal_error_casting_vchar_to_array branch August 20, 2025 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants