Skip to content

[doc](variant) Rework the VARIANT reference for write, CAST, NULL, ALTER and ordering semantics - #4170

Merged
eldenmoon merged 8 commits into
apache:masterfrom
eldenmoon:docs/variant-behavior-refresh
Oct 10, 2026
Merged

eldenmoon merged 8 commits into
apache:masterfrom
eldenmoon:docs/variant-behavior-refresh

Conversation

@eldenmoon

@eldenmoon eldenmoon commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Draft until apache/doris#67675 is merged. That PR introduces the whole-VARIANT =/!=/<=>, joins, ORDER BY and window keys, and the rejection of invalid UTF-8 in CAST(string AS VARIANT); it removes the Schema Template auto cast and no longer accepts TIMESTAMPTZ Schema Template paths. Everything else on these pages describes current master.

What changed

Reworks the dev (Doris 5.0.0) VARIANT documentation, English and Chinese, so that it describes the behavior of master together with apache/doris#67675:

  • VARIANT reference (VARIANT.md), still one page for the type:
    • Write data: INSERT and CAST keep strings, while PARSE_TO_VARIANT and load jobs parse JSON. Parse errors: out-of-range numbers turn the whole document into a string, plus the depth limit, invalid UTF-8, duplicate keys, and the BE configurations that change these rules.
    • What storage keeps: null members and empty containers outside arrays are dropped, while values inside arrays are kept; a root null reads back as {}; dates outside a Schema Template become strings; dotted keys become nested paths; mixed booleans and numbers can become 1/0.
    • Type inference and type conflicts: inference and storage types for the current storage format. For example, integers mixed with doubles and arrays of objects are stored as JSONB.
    • CAST:
      • CAST in both directions, with the whitelists verified;
      • a failed conversion returns NULL regardless of enable_strict_cast, while a type pair without any conversion fails the statement.
    • NULL semantics: SQL NULL versus VARIANT null for the common operators, and how storage removes null members.
    • Comparison, grouping, and ordering:
      • the supported-operation matrix;
      • equality rules, for example 1 = 1.0 but DECIMAL 1.5 != DOUBLE 1.5;
      • the total order across kinds;
      • typed comparison versus VARIANT comparison;
      • caveats: cost, no runtime filter, and the effect of stored values.
    • Schema Template: accepted field types; write-time conversion; reading without auto cast; glob matching.
    • ALTER TABLE:
      • ADD COLUMN ... VARIANT NOT NULL is impossible;
      • a column with a Schema Template only allows DROP, RENAME and a comment change;
      • of the properties, only variant_doc_materialization_min_rows can change;
      • workarounds for the rest.
    • Column properties, session variables, BE configuration: defaults, ranges, and whether ALTER can change each one.
  • Function pages:
    • PARSE_TO_VARIANT / TRY_PARSE_TO_VARIANT: which inputs error or return NULL, and which are kept as strings.
    • VARIANT_TYPE: returns one type name for the value itself.
    • ELEMENT_AT: the NULL cases and null semantics.
  • Workload guide, import page, feature page:
    • no example INSERTs a JSON string into VARIANT any more;
    • filters CAST paths to their stored type;
    • the import page shows how storage normalizes values, with a small demo, and DESC output regenerated from its own sample;
    • the feature page drops outdated claims about join keys and type promotion.
  • Links: every cross-page link is now a file link (.md/.mdx). The site sets trailingSlash: true, so extension-less relative links resolve under the current page. For example, the published VARIANT page currently links to .../semi-structured/VARIANT/variant-workload-guide/.

Why

The dev pages still described several 4.x behaviors that no longer hold on master, and some pages contradicted each other:

  • The examples wrote JSON with INSERT ... VALUES ('{...}'), which now stores a VARIANT string.
  • The FAQ said VARIANT null and SQL NULL are equivalent.
  • TRY_PARSE_TO_VARIANT claimed that invalid JSON returns NULL.
  • VARIANT_TYPE showed a per-path JSON map.
  • The CAST whitelist listed TIMESTAMPTZ as a source.
  • The type promotion table was outdated.
  • The index examples used comparisons that cannot use the index.

CAST behavior, ALTER support, NULL handling and GROUP BY/ORDER BY semantics were not documented at all.

Version wording: "Doris 4.2 and later" is replaced by "Doris 5.0.0 and later", because branch-4.2 keeps enable_variant_v2 = false (VARIANT V1 by default), and other dev pages already refer to master features as 5.0.0. The 4.x pages are intentionally unchanged.

Validation

  • Every SQL example, every shown output, and every behavior claim was run on a single-node cluster built from the [feature](variant) Support relational operations for Variant doris#67675 head (master merged at 050b83b148c), with default session variables and BE configuration. Coverage:
    • INSERT, group commit, http_stream, and Stream Load in CSV and JSON format;
    • parse errors;
    • CAST with strict mode on and off;
    • null handling before and after storage;
    • ORDER BY, GROUP BY, join, IN, window and conditional-expression results;
    • every ALTER case on the page;
    • Schema Template conversions.
  • The code paths were cross-checked:
    • FE: TypeCoercionUtils, CheckCast, Column.checkSchemaChangeAllowed, PropertyAnalyzer, NereidsLoadUtils, LogicalPlanBuilder;
    • BE: variant_canonical.cpp, variant_string_parse.cpp, variant_path_builder.cpp, new_json_reader.cpp.
  • Two independent reviews ran on the first revision: a technical check against the code and the cluster, and an English/Chinese consistency and documentation-quality check. Their findings are addressed in the second commit. A third review of that revision found more mismatches with the engine, addressed in the third commit: CAST errors for unsupported type pairs, template writes, group commit through prepared statements, Arrow, and subpath aliases.
  • The fourth commit addresses the review comments on this PR. It removes the write-path table, the implicit conversion subsection and the computed-versus-stored subsection, shortens NULL semantics and ALTER TABLE, and adds a normalization demo to the import page. It also restores five pages (the feature page, ELEMENT_AT, TRY_PARSE_TO_VARIANT) that the third commit had reverted to master by mistake. Another independent review of that commit found a list-rendering bug and a few imprecise sentences, fixed in the fifth commit.
  • The sixth commit follows [feature](variant) Support relational operations for Variant doris#67675, which now rejects TIMESTAMPTZ Schema Template paths, and narrows the note on dates stored as text: that happens to root values and to paths that also hold other types. Object paths that hold only dates keep their type in subcolumns, sparse columns and DOC mode; verified on the cluster, including after a full compaction.
  • git diff --check passed.
  • docs:links:changed, docs:sql-functions:changed, docs:features:changed, and the frontmatter and markdown-structure lints report no findings.
  • docs:i18n-sync:changed reports only i18n-sync-version-counterpart warnings. This is an intentional exception, because the 4.x docs describe VARIANT V1.
  • docs:seo:changed reports only the existing cross-version duplicate-title warnings.

Behaviors that look like bugs are documented as they are today and tracked separately:

  • valid JSON with out-of-range numbers becomes a string;
  • implicit DECIMAL(38, 9) comparisons;
  • a Schema Template date, time, or IP path drops valid strings when a batch mixes strings and numbers, and fails the whole write when it receives only numbers;
  • CAST from VARIANT fails, instead of returning NULL, for type pairs without a conversion, such as a number to TIMESTAMPTZ or IPV4;

The pages will need updating if these are fixed.

🤖 Generated with Claude Code

…TER and ordering semantics

Rewrite the dev (5.0.0) VARIANT reference in English and Chinese around the
behavior of master with apache/doris#67675:

- how each write path turns input into a VARIANT value (INSERT keeps strings,
  PARSE_TO_VARIANT and load jobs parse JSON, invalid JSON stays a string);
- what storage normalizes (null members, empty containers, root null, dates,
  dotted keys, mixed booleans and numbers);
- CAST in both directions, failure to NULL regardless of strict mode, and the
  implicit conversion rules for sub-path comparisons, function and JSON
  function arguments, with no Schema Template auto cast;
- SQL NULL versus VARIANT null, for computed and for stored values;
- equality, hashing and the total order used by GROUP BY, DISTINCT, joins,
  ORDER BY and window keys, with the supported-operation matrix and caveats;
- Schema Template write conversion, reading and pattern matching;
- storage type inference and conflicts;
- ALTER TABLE support (no NOT NULL ADD COLUMN, no template or property
  change except variant_doc_materialization_min_rows);
- column properties, session variables and BE configurations.

Also fix PARSE_TO_VARIANT, TRY_PARSE_TO_VARIANT, VARIANT_TYPE and ELEMENT_AT,
the workload guide and import examples, and the VARIANT feature page, which
still inserted JSON strings or described 4.x behavior.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@eldenmoon
eldenmoon marked this pull request as draft September 23, 2026 05:37
Fix what two independent reviews and further tests on a PR #67675 cluster
found in the first revision:

- make every cross-page link a file link (.md/.mdx); with trailingSlash
  enabled, extension-less relative links resolve under the current page and
  already break on the published site;
- document the whole-document fallback to a string when JSON contains an
  integer outside [-2^63, 2^64 - 1] or a number outside the DOUBLE range,
  plus the 128-level depth limit, invalid UTF-8, and the duplicate-key and
  invalid-JSON BE configurations;
- document the implicit conversion pitfalls: DECIMAL(38, 9) overflow and
  rounding, no index or zone-map use, VARIANT mixed with other types in
  IF/CASE/COALESCE/IFNULL/UNION, and `v IN (literal list)`;
- INSERT ... SELECT from table functions and http_stream keep strings; load
  jobs parse strings of any format; top-level JSON booleans load as 1/0;
- storage normalization applies to object members outside arrays, all-null
  arrays can be dropped, and dotted-key conflicts fail the whole write;
- ALTER on a column with a Schema Template only allows DROP, RENAME and a
  comment change; DOC mode template types; TIMESTAMP_NS text; LARGEINT and
  UUID in CAST; VARIANT_TYPE timestamp names;
- use explicit CAST in index examples, fix FAQ 5 and the TRY_PARSE_TO_VARIANT
  description, regenerate the import page DESC output, and clean up the old
  sections (version notes, terminology, table pipe escaping).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…review

A second independent review of the reworked pages, checked on a PR #67675
cluster, found statements that still did not match the engine:

- CAST from VARIANT returns NULL only when the value's type has a conversion
  to the target; a number to TIMESTAMPTZ/IPV4/IPV6 or a date to BOOLEAN fails
  the statement, and TIMESTAMPTZ does not accept numbers;
- a JSON number on an IPV4/IPV6/TIMESTAMPTZ template path fails the whole
  write, and mixing strings and numbers on date, time or IP template paths can
  drop valid values (caution added);
- group commit INSERT through JDBC server-side prepared statements parses
  JSON like a load job; the Arrow format cannot load VARIANT; filtered NOT NULL
  rows count toward max_filter_ratio;
- an alias of a subpath is a whole value for implicit conversion; TIMESTAMPTZ
  compares in the session time zone; only comparisons whose implicit type
  differs from the stored type lose index use;
- the JSON workaround turns 39-digit integers into strings; deeply nested
  documents rejected by the parser are kept as strings; the duplicate-path
  configuration also covers dotted-key conflicts;
- fix FAQ 6, the zh rounding wording and link text, VARIANT_TYPE for
  LARGEINT, the workload guide ALTER note, and the zh import description.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

## 导入时值如何转换

- **CSV 格式:** `VARIANT` 字段的文本按 JSON 解析。`\N` 导入为 SQL `NULL`。非法 JSON 文本导入为 VARIANT 字符串。

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CSV 格式、JSON 格式、超范围的数值 不用提及

- **超范围的数值:** 包含超出 [-2^63, 2^64 - 1] 的整数或超出 `DOUBLE` 范围的数值的文档无法解析,整个文档会作为一个 VARIANT 字符串导入。
- **`NOT NULL` 列:** VARIANT 值为 SQL `NULL` 的行会被过滤,包括解析失败而得到 SQL `NULL` 的值。被过滤的行计入 `max_filter_ratio`(默认 `0`),因此默认情况下导入作业会失败。
- **INSERT 的行为不同:** `INSERT` 不解析字符串。`INSERT INTO t VALUES (1, '{"a": 1}')` 写入的是 VARIANT 字符串 `{"a": 1}`,从 `s3()`、`hdfs()`、`local()` 执行 `INSERT INTO ... SELECT` 也是如此。需要写入对象时请使用 `PARSE_TO_VARIANT`。
- **存储会规范化值:** 值为 `null` 的对象成员,以及值为空对象或空数组的成员,在写入时可能被移除。

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

存储会规范化值 加个简单的 demo


- `INSERT` 会把字符串作为 VARIANT 字符串写入,不再按 JSON 解析,从 `s3()`、`hdfs()` 等表函数执行 `INSERT INTO ... SELECT` 也是如此。用 `INSERT` 写入 JSON 文本时请使用 `PARSE_TO_VARIANT`;Stream Load 等导入作业仍会解析 JSON。
- 整个 VARIANT 值支持 `GROUP BY`、`DISTINCT` 和集合运算;两个 VARIANT 值之间可以用 `=`、`!=`、`<=>` 比较,也可以作为 Join 键、`ORDER BY` 键和窗口键。
- 即使路径在 Schema Template 中声明了类型,`v['path']` 仍是 `VARIANT` 类型,需要显式 CAST。会话变量 `enable_variant_schema_auto_cast` 已不再生效。

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“即使路径在 Schema Template 中声明了类型” 这段不用描述


## 写入数据 {#write-data}

### 输入如何变成 VARIANT 值 {#how-input-becomes-a-variant-value}

@eldenmoon eldenmoon Sep 23, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“输入如何变成 VARIANT 值 ”, 这段不保留删了,前面重复了


- 如果要将字符串或 JSON/JSONB 表达式解析为结构化 VARIANT 值,请使用 [PARSE_TO_VARIANT](../../../sql-functions/scalar-functions/variant-functions/parse-to-variant)。
- 如果要将受支持的 SQL 值转换为 VARIANT,请使用 `CAST(expression AS VARIANT)`。字符串会保留为 VARIANT 字符串值,该 CAST 不解析 JSON。
### 解析错误 {#parse-errors}

@eldenmoon eldenmoon Sep 23, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里你再理一下正确行为应该是怎么样的,这里的行为比较奇怪, 这些 BE 的 config 看下是不是可以淘汰了

| 数组内的值,如 `{"arr": [{"a": null}, {}, [], null]}` | 原样保留;`v['arr'][1]['a']` 是 VARIANT `null`。 |
| 根值为 JSON `null`,如 `PARSE_TO_VARIANT('null')` | 空对象 `{}` |
| 根值为空数组 `[]` 或空对象 `{}` | 保留 |
| 不在 Schema Template 路径上的 `DATE`、`DATETIME` 值,如 `CAST(date_col AS VARIANT)` | 以文本形式存储,读回时是字符串 |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为什么读回时是字符串, parquet 编码没有这个类型?

| `BIGINT` 范围内的整数 | `tinyint`、`smallint`、`int` 或 `bigint` 中能容纳该值的最小类型 |
| 超出 `BIGINT` 范围、不超过 18446744073709551615 的整数 | `decimal` |
| 带小数部分或指数的数值 | `double` |
| 超出 [-2^63, 2^64 - 1] 的整数,或超出 `DOUBLE` 范围的数值 | 不作为数值处理:整段文本都是非法 JSON(参见[解析错误](#parse-errors)) |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里如果要支持?应该怎么做?或者加上引号比如"12345"?

```

### Decimal 与日期时间转换限制
读取整个 VARIANT 值会返回 JSON 文本。对象 key 按字节序输出,且不含空白,因此与输入文本并非按字节完全一致。字符串根值输出时不带引号。`CAST(v AS STRING)` 则按对应 SQL 类型的格式输出标量根值,例如布尔根值变为 `1` 或 `0`。

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"CAST(v AS STRING) 则按对应 SQL 类型的格式输出标量根值,例如布尔根值变为 1 或 0" 这句话不用写

| `CHAR`、`VARCHAR`、`STRING` | VARIANT 字符串,不解析 JSON 文本。字符串必须是合法的 UTF-8,否则 CAST 报错。 |
| `BOOLEAN` | 布尔值。 |
| `TINYINT`、`SMALLINT`、`INT`、`BIGINT` | 整数。 |
| `LARGEINT` | 定点数。绝对值超过 10^38 - 1 的值会变成字符串。 |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“绝对值超过 10^38 - 1 的值会变成字符串” 这是正确的行为吗?应该怎么处理

| `BOOLEAN` | 布尔值保持不变;数值非零即为 `true`;字符串按 `CAST(string AS BOOLEAN)` 的规则转换。 |
| `TINYINT`、`SMALLINT`、`INT`、`BIGINT`、`LARGEINT` | 整数。定点数和浮点数的小数部分被截断(`1.5` 变为 `1`);布尔值变为 `1` 或 `0`;`"123"` 这样的数字字符串会被转换。 |
| `FLOAT`、`DOUBLE`、`DECIMAL(p, s)` | 数值和数字字符串。`DECIMAL` 按 scale `s` 舍入。超出 `FLOAT` 范围的值变为 `Infinity`。 |
| `DATE`、`DATETIME(p)`、`TIMESTAMP_NS` | 日期时间值、日期时间格式的字符串,以及 `20240102` 这样的数值。 |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同上?这里如果要支持怎么处理

| `DATE` | 不含时间和时区的日历日期。 |
| `DATETIME(p)` | `0 <= p <= 6`,不做时区调整。 |
| `TIMESTAMP_NS` | 纳秒精度,不做时区调整;值必须在 TIMESTAMP_NS 取值范围内。 |
| `TIMESTAMPTZ(p)` | 不能 CAST 为 VARIANT。可以在 Schema Template 中把路径声明为 `TIMESTAMPTZ`。 |

@eldenmoon eldenmoon Sep 23, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema Template 禁止一下TIMESTAMPTZ,直接不支持

## NULL 语义 {#null-semantics}

这些规则适用于 `GROUP BY`、`DISTINCT`、`COUNT(DISTINCT ...)`、`INTERSECT`、`EXCEPT` 和 `UNION DISTINCT` 等已支持的操作,但不会开放根 VARIANT 比较谓词:直接执行 `VARIANT = VARIANT` 或排序比较仍不支持。
### SQL NULL 与 VARIANT null {#sql-null-and-variant-null}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这一节写的比较复杂, 精简写


如果 `COALESCE` 或 `IFNULL` 中的 `x` 不是 VARIANT,VARIANT 一侧会先被 CAST,参见[隐式转换](#implicit-conversion)。

### 计算侧的值与存储侧的值 {#computed-values-and-stored-values}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"计算侧的值与存储侧的值" 这一节删掉, 前面提过了

| `TIMESTAMPTZ(p)` | 不能 CAST 为 VARIANT。可以在 Schema Template 中把路径声明为 `TIMESTAMPTZ`。 |
| `TIME` | 不支持。 |

### 隐式转换 {#implicit-conversion}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

隐式转换 这一节删掉

| {"datetime_val":"datetimev2","decimal_val":"decimal128i","ip_val":"ipv4","large_int_val":"largeint","string_val":"string","tz_val":"timestamptz"} |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
```
`CHAR`、`VARCHAR`、`DECIMALV2`、`TIME`、`JSON`、`MAP`、`STRUCT` 和嵌套数组不能用于 Schema Template。DOC mode 下,模板字段进一步限制为字符串、整数、`FLOAT`、`DOUBLE`、`BOOLEAN` 以及由这些类型组成的数组。

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

550 这一行删除

列创建后不能修改 Schema Template,参见 [ALTER TABLE](#alter-table)。

当同一路径出现不兼容类型(如同一字段既出现整数又出现字符串)时,将提升为 JSONB 类型以避免信息丢失:
## ALTER TABLE {#alter-table}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这一节简化一下

- 输入为 JSON 字面量 `null` 时返回 Variant/JSON `null`,它与 SQL `NULL` 不同。
- JSON 非法、对象 key 重复且当前校验设置不允许、嵌套深度超限或发生其他校验错误时,查询失败。
- 输入为 JSON 字面量 `null` 时返回 VARIANT `null`,它与 SQL `NULL` 不同。
- 非法 JSON 文本会作为 VARIANT 字符串返回;包含超出 [-2^63, 2^64 - 1] 的整数或超出 `DOUBLE` 范围的数值的 JSON 也是如此,整段文本变为一个字符串。该行为由 BE 配置 `variant_throw_exeception_on_invalid_json`(默认 `false`)控制;设置为 `true` 后,这类输入会使查询失败。

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variant_throw_exeception_on_invalid_json 这个配置是否不应该保留, 有 try_parse_to_variant/parse_to_variant 本身控制, 比如 try 的不报错,否则报错?

eldenmoon and others added 2 commits September 23, 2026 15:55
- VARIANT reference: drop the Schema Template auto-cast note, the
  input-to-VARIANT table, the implicit conversion subsection, the
  computed-vs-stored NULL subsection and the unsupported template types
  line; shorten NULL semantics and ALTER TABLE; rename the CAST section.
- Import page: drop the CSV, JSON and out-of-range number bullets and add
  a demo of how storage removes null members and empty containers.
- Restore the key-features page, ELEMENT_AT and TRY_PARSE_TO_VARIANT
  changes that 587127d reverted by mistake.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Import page: separate the normalization paragraph from the list so it
  renders as its own paragraph, rename the section to match what is left,
  and point to Parse errors instead of the removed write rules.
- NULL semantics and FAQ: the null-member removal applies only outside
  arrays.
- Mixing VARIANT with an integer converts to DECIMAL(38, 9); say so where
  the page mentions the conversion, which FAQ 6 relies on.
- ALTER TABLE: only MODIFY COLUMN is restricted on a column that has a
  Schema Template; restore the rule for omitted properties; name the only
  property that can change.
- ELEMENT_AT: move the null note below the index example it interrupted.
- TRY_PARSE_TO_VARIANT: add the duplicate-key configuration and the empty
  string case, as on the PARSE_TO_VARIANT page.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Sep 23, 2026
…plates

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: A Schema Template path declared as TIMESTAMPTZ is only half
supported: CAST(TIMESTAMPTZ AS VARIANT) is rejected, DOC mode rejects the
path, and a JSON number written to the path fails the whole write.

Remove TIMESTAMP_TZ from the VARIANT sub-type whitelist. CREATE TABLE and
ALTER TABLE ... ADD COLUMN now reject VARIANT<'p': TIMESTAMPTZ> and
VARIANT<'p': ARRAY<TIMESTAMPTZ>> with "VARIANT unsupported sub-type". The
whitelist is only checked when a type is defined, so tables that already
declare such a path (branch-4.0 and branch-4.1 accept it) keep working.

test_variant_timestamptz_sparse and the TIMESTAMPTZ case of
test_variant_json_functions can no longer create their tables and are
removed. The sparse binary round trip they covered is still tested by
DataTypeTimeStampTzSerDeTest.binary_roundtrip.

### Release note

A VARIANT Schema Template can no longer declare TIMESTAMPTZ paths. Existing
tables are not affected.

### Check List (For Author)

- Test: Regression test (test_predefine_ddl, test_variant_json_functions) and
  Unit Test (TypeTest)
- Behavior changed: Yes (CREATE/ALTER with a TIMESTAMPTZ template path fails)
- Does this need documentation: Yes (apache/doris-website#4170)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…d as text

- apache/doris#67675 now rejects TIMESTAMPTZ in a Schema Template; remove
  it from the accepted types, the example and the write caution, and say so
  in the date/time conversion table.
- Dates become text only in the root value and on paths that also hold
  other types. An object path that holds only dates keeps them as dates in
  subcolumns, sparse columns and DOC mode, also after compaction.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

## Using VARIANT
:::info Version
This page describes VARIANT in Doris 5.0.0 and later. The differences from Doris 4.x that are most likely to affect existing SQL:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我们是不是4.2 就是这样了?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4.2 上说明一下开关, 5.0 默认打开

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4.2 默认还是关闭

### 排序规则 {#ordering}

SQL `NULL` 与 Variant/JSON `null` 是不同的值:
对 VARIANT 值执行 `ORDER BY` 时使用统一的全序。不同种类的值先按种类排序:

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里再补充一下number的比较规则再细化一下

- Variant/JSON `null` 是一个 VARIANT 值,例如 `PARSE_TO_VARIANT('null')` 的返回值。
- `TRY_PARSE_TO_VARIANT` 遇到非法输入时返回 SQL `NULL`,这与成功解析 JSON 字面量 `null` 不同。
```text
null < boolean < number < string < binary < date < timestamp with time zone

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

把number 里float,int,decimal 具体怎么比较的再详细说一下吧

Explain integral normalization, exact mixed numeric comparison, and decimal
versus floating-point tie-breaking in the English and Chinese VARIANT
ordering sections. Include 1/1.0/1.00, fractional values, decimal scales,
negative zero, floating-point precision, and a runnable ordering example.

Validation: execute the new sorting example and eight equality checks on
existing PR-compatible binaries; verify comparison code against #67675;
pass diff, markdown structure, frontmatter and changed-link checks.
@eldenmoon
eldenmoon merged commit d108285 into apache:master Oct 10, 2026
3 checks passed

This branch was previously deployed

1 inactive deployment
Production — bd57ff2e Deployed Oct 10, 2026 by eldenmoon via Build Check #8473
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants