Skip to content

feat(sql): allow underscore number separator in TICK syntax#6749

Merged
bluestreak01 merged 4 commits intomasterfrom
mt_tick-underscore
Feb 5, 2026
Merged

feat(sql): allow underscore number separator in TICK syntax#6749
bluestreak01 merged 4 commits intomasterfrom
mt_tick-underscore

Conversation

@mtopolnik
Copy link
Copy Markdown
Contributor

Summary

  • Allow underscore as a number separator in TICK date expressions (e.g., $now-10_000T) and duration suffixes (e.g., ;1_500T), consistent with Numbers.parseInt() which already supports this
  • Improve error position reporting in DateExpressionEvaluator and IntervalUtils.expandDateVariableRange to point at the specific offending location instead of the start of the expression

Closes #6736

Test plan

  • New tests for underscore in date variable expressions (basic, compact, range, business days)
  • New test for underscore in duration suffixes
  • New tests for invalid underscore placement (leading, trailing, consecutive)
  • All 824 TickExprTest cases pass

🤖 Generated with Claude Code

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Feb 5, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • ✅ Full review completed - (🔄 Check again to review again)
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch mt_tick-underscore

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 16 / 16 (100.00%)

file detail

path covered line new line coverage
🔵 io/questdb/griffin/model/IntervalUtils.java 3 3 100.00%
🔵 io/questdb/griffin/model/DateExpressionEvaluator.java 13 13 100.00%

@bluestreak01
Copy link
Copy Markdown
Member

@CodeRabbit pls review critically, don't hallucinate and double-check each point

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Feb 5, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

I'll conduct a thorough critical review of this PR. Let me start by examining the actual code changes and test coverage.


🏁 Script executed:

#!/bin/bash
# Get the PR diff and list of changed files
gh pr view 6749 --json files --jq '.files[] | .path' | head -20

# Get the full diff to see what changed
gh pr diff 6749

Length of output: 10397


Let me examine the Numbers.parseInt implementation to verify the claim about underscore support, and check for potential edge cases.


🏁 Script executed:

#!/bin/bash
# Find and read the Numbers.parseInt implementation to verify underscore support
fd -t f "Numbers.java" | head -5

Length of output: 46


🏁 Script executed:

#!/bin/bash
# Read the parseInt method implementation to verify underscore support
cat core/src/main/java/io/questdb/std/Numbers.java | grep -A 50 "parseInt" | head -100

Length of output: 3358


🏁 Script executed:

#!/bin/bash
# Look for the parseInt0 implementation which is the actual implementation
rg -A 30 "private static int parseInt0" core/src/main/java/io/questdb/std/Numbers.java

Length of output: 1228


🏁 Script executed:

#!/bin/bash
# Get the complete parseInt0 implementation
rg -A 50 "private static int parseInt0" core/src/main/java/io/questdb/std/Numbers.java | head -70

Length of output: 1985


🏁 Script executed:

#!/bin/bash
# Check the complete addDuration implementation to see how it parses the number
rg -B 10 -A 40 "private static long addDuration" core/src/main/java/io/questdb/griffin/model/IntervalUtils.java

Length of output: 1944


🏁 Script executed:

#!/bin/bash
# Check if there are other places that parse tick expressions that might need underscore support
rg -n "parseInt.*[Tt]ick|[Tt]ick.*parseInt" --type java | head -20

# Also search for other number parsing in interval/date related code
rg -n "Character\.isDigit|>= '0' && .* <= '9'" core/src/main/java/io/questdb/griffin/model/ | head -30

Length of output: 1091


🏁 Script executed:

#!/bin/bash
# Check what those other number parsing locations at lines 1847 and 1882 are for
# They don't support underscores
sed -n '1830,1900p' core/src/main/java/io/questdb/griffin/model/IntervalUtils.java

Length of output: 2845


🏁 Script executed:

#!/bin/bash
# Check if bracket expansion like [1000,2000] should support underscores
# First, let's understand what bracket expansion is used for
rg -B 5 -A 3 "bracket expansion" core/src/test/java/io/questdb/test/griffin/model/TickExprTest.java | head -40

Length of output: 1294


@bluestreak01 I've conducted a thorough critical review. Here are my findings:

Confirmed correct:

  1. Underscore support claim verified: Numbers.parseInt() does indeed support underscores with proper validation (rejects leading, trailing, and consecutive underscores via digitCounter tracking)

  2. Error position improvements are mathematically sound: The calculations like errorPos + offsetStart - lo correctly convert relative positions within the expression to absolute SQL positions for error reporting

  3. Test coverage is comprehensive: Covers valid cases (basic, compact, ranges, business days, durations) and invalid cases (leading, trailing, consecutive underscores)

Critical issues found:

1. Scope creep: Unrelated cosmetic refactoring

Lines 89-95 in DateExpressionEvaluator.java:

-        while (varEnd < hi) {
+        for (; varEnd < hi; varEnd++) {
             char c = expression.charAt(varEnd);
             if (c == ' ' || c == '+' || c == '-') {
                 break;
             }
-            varEnd++;
         }

This while→for conversion is purely cosmetic and unrelated to underscore support. It's not mentioned in the PR description and doesn't add value. The second while→for conversion (lines 125-132) IS justified since it's part of number parsing with underscores, but this first one is not.

Question: Why is this change included? It adds unnecessary noise to the diff.

2. Inconsistent underscore support across parsing contexts

You added underscore support to:

  • ✅ Date expression numbers: $now - 10_000T
  • ✅ Duration suffixes: ;1_500T

But NOT to:

  • ❌ Bracket expansion: 2018-01-[10_000,20_000] (lines 1847, 1882 in IntervalUtils)

The bracket expansion still uses Character.isDigit() without underscore support. This creates inconsistency: users can write 2018-01-10;10_000T but NOT 2018-01-[10_000,20_000].

Questions:

  • Is this intentional?
  • Should bracket expansion also support underscores for consistency?
  • If not, is there a principled reason why TICK syntax supports it but bracket expansion doesn't?

3. Edge case: What about negative numbers with underscores?

The PR doesn't test expressions like $now - -1_000T (double negative). While this might be nonsensical, should it be explicitly rejected with a clear error? The current code would fail at the operator parsing stage, but I don't see a test confirming the error message quality.

4. Minor: CLAUDE.md change seems unrelated

The change to CLAUDE.md about alphabetical sorting clarification appears unrelated to this PR's objective. Should this be in a separate commit/PR?

🤔 Design questions:

  1. Why allow underscores at all? The PR says "consistent with Numbers.parseInt()", but that's a circular argument. What's the actual user benefit? Are there queries with large tick values where readability matters? (e.g., $now - 1_000_000T vs $now - 1000000T)

  2. Performance: The validation now happens twice—once in the manual loop checking c == '_' || isDigit(c), then again in Numbers.parseInt(). Why not just call parseInt() on a broader range and let it handle all validation? (Minor, but worth considering)

  3. Error messages: When parseInt fails on invalid underscores, users get "Invalid number in date expression". Would "Invalid underscore placement" be clearer?

📊 Verdict:

The core underscore support is correctly implemented and well-tested. However:

  • Remove the unrelated cosmetic while→for change (lines 89-95)
  • Address the inconsistency with bracket expansion or document why it's intentional
  • Consider splitting CLAUDE.md into a separate PR

The implementation correctly delegates validation to Numbers.parseInt(), and error positions are properly calculated. With the scope creep removed, this would be a solid PR.


🧠 Learnings used
Learnt from: jerrinot
Repo: questdb/questdb PR: 0
File: :0-0
Timestamp: 2026-02-04T11:18:36.492Z
Learning: For PRs by jerrinot, always use a thorough and critical review mode: question design decisions, challenge assumptions, scrutinize implementations for edge cases and potential issues, and avoid superficial validation. They prefer hard-core reviews that push back on the code.

@bluestreak01 bluestreak01 merged commit 1e46e66 into master Feb 5, 2026
44 checks passed
@bluestreak01 bluestreak01 deleted the mt_tick-underscore branch February 5, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TICK syntax should allow _ number separator

3 participants