Skip to content

[Feature](function) Support function DEFAULT - #57531

Merged
zclllyybb merged 10 commits into
apache:masterfrom
linrrzqqq:default
Jan 23, 2026
Merged

zclllyybb merged 10 commits into
apache:masterfrom
linrrzqqq:default

Conversation

@linrrzqqq

@linrrzqqq linrrzqqq commented Oct 30, 2025 •

Copy link
Copy Markdown
Collaborator

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

Return the default value of column

CREATE TABLE test_default_scalar (
    c_id INT,
    c_bool                   BOOLEAN                               NULL     DEFAULT 1,
    c_tinyint                TINYINT                               NULL     DEFAULT 7,
    c_smallint               SMALLINT                              NULL     DEFAULT 32000,
    c_int                    INT                                   NULL     DEFAULT 2147483647,
    c_bigint                 BIGINT                                NULL     DEFAULT 9223372036854775807,
    c_largeint               LARGEINT                              NULL     DEFAULT '170141183460469231731687303715884105727',
    c_float                  FLOAT                                 NULL     DEFAULT 3.125,
    c_double                 DOUBLE                                NULL     DEFAULT 2.718281828,
    c_decimal                DECIMAL(27, 9)                        NULL     DEFAULT '123456789.123456789',
    c_decimal_compact        DECIMAL(18, 4)                        NULL     DEFAULT '99999.1234',
    c_pi                     DOUBLE                                NULL     DEFAULT PI,
    c_e                      DOUBLE                                NULL     DEFAULT E,
    c_char                   CHAR(8)                               NULL     DEFAULT 'charDemo',
    c_varchar                VARCHAR(32)                           NULL     DEFAULT '',
    c_string                 STRING                                NULL     DEFAULT 'plain string',
    c_datetime               DATETIME                              NULL     DEFAULT '2025-10-25 11:22:33',
    c_date                   DATE                                  NULL     DEFAULT '2025-10-31',
    c_json                   JSON                                  NULL,
    c_ipv4                   IPV4                                  NULL     DEFAULT '192.168.1.1',
    c_ipv6                   IPV6                                  NULL     DEFAULT '2001:db8::1',
    c_array_int              ARRAY<INT>                            NULL     DEFAULT '[]',
    c_array_string           ARRAY<STRING>                         NULL,
    c_map_str_int            MAP<STRING, INT>                      NULL,
    c_struct                 STRUCT<f1:INT,f2:STRING,f3:BOOLEAN>   NULL,
    c_variant                VARIANT                               NULL
) PROPERTIES ( 'replication_num' = '1');

SELECT
    default(c_id)                   AS default_id,
    default(c_bool)                 AS default_c_bool,
    default(c_tinyint)              AS default_c_tinyint,
    default(c_smallint)             AS default_c_smallint,
    default(c_int)                  AS default_c_int,
    default(c_bigint)               AS default_c_bigint,
    default(c_largeint)             AS default_c_largeint,
    default(c_float)                AS default_c_float,
    default(c_double)               AS default_c_double,
    default(c_decimal)              AS default_c_decimal,
    default(c_decimal_compact)      AS default_c_decimal_compact,
    default(c_char)                 AS default_c_char,
    default(c_varchar)              AS default_c_varchar,
    default(c_string)               AS default_c_string,
    default(c_datetime)             AS default_datetime,
    default(c_date)                 AS defaut_time,
    default(c_json)                 AS default_c_json,
    default(c_ipv4)                 AS default_c_ipv4,
    default(c_ipv6)                 AS default_c_ipv6,
    default(c_array_int)            AS default_c_array_int,
    default(c_array_string)         AS default_c_array_string,
    default(c_map_str_int)          AS default_c_map_str_int,
    default(c_struct)               AS default_c_struct,
    default(c_variant)              AS default_c_variant
FROM test_default_scalar
LIMIT 1;
+------------+----------------+-------------------+--------------------+---------------+---------------------+-----------------------------------------+-----------------+------------------+---------------------+---------------------------+----------------+-------------------+------------------+---------------------+-------------+----------------+----------------+----------------+---------------------+------------------------+-----------------------+------------------+-------------------+
| default_id | default_c_bool | default_c_tinyint | default_c_smallint | default_c_int | default_c_bigint    | default_c_largeint                      | default_c_float | default_c_double | default_c_decimal   | default_c_decimal_compact | default_c_char | default_c_varchar | default_c_string | default_datetime    | defaut_time | default_c_json | default_c_ipv4 | default_c_ipv6 | default_c_array_int | default_c_array_string | default_c_map_str_int | default_c_struct | default_c_variant |
+------------+----------------+-------------------+--------------------+---------------+---------------------+-----------------------------------------+-----------------+------------------+---------------------+---------------------------+----------------+-------------------+------------------+---------------------+-------------+----------------+----------------+----------------+---------------------+------------------------+-----------------------+------------------+-------------------+
|       NULL |              1 |                 7 |              32000 |    2147483647 | 9223372036854775807 | 170141183460469231731687303715884105727 |           3.125 |      2.718281828 | 123456789.123456789 |                99999.1234 | charDemo       |                   | plain string     | 2025-10-25 11:22:33 | 2025-10-31  | NULL           | 192.168.1.1    | 2001:db8::1    | []                  | NULL                   | NULL                  | NULL             | NULL              |
+------------+----------------+-------------------+--------------------+---------------+---------------------+-----------------------------------------+-----------------+------------------+---------------------+---------------------------+----------------+-------------------+------------------+---------------------+-------------+----------------+----------------+----------------+---------------------+------------------------+-----------------------+------------------+-------------------+

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas

Thearas commented Oct 30, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@linrrzqqq
linrrzqqq force-pushed the default branch 2 times, most recently from 0b10e76 to 5ca6a29 Compare October 30, 2025 14:38
@linrrzqqq

Copy link
Copy Markdown
Collaborator Author

run buildall

@linrrzqqq
linrrzqqq marked this pull request as ready for review October 30, 2025 14:40
@zclllyybb zclllyybb self-assigned this Oct 30, 2025
@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 5.88% (1/17) 🎉
Increment coverage report
Complete coverage report

@doris-robot

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.57 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5ca6a29e47eda34b1a634fcd82b0dd56e81528eb, data reload: false

query1	0.06	0.06	0.06
query2	0.12	0.06	0.07
query3	0.26	0.09	0.09
query4	1.61	0.12	0.12
query5	0.30	0.28	0.26
query6	1.17	0.69	0.69
query7	0.04	0.03	0.02
query8	0.07	0.05	0.05
query9	0.67	0.57	0.58
query10	0.63	0.62	0.63
query11	0.20	0.13	0.14
query12	0.18	0.14	0.14
query13	0.66	0.62	0.61
query14	1.05	1.02	1.05
query15	0.92	0.86	0.88
query16	0.42	0.44	0.43
query17	1.17	1.23	1.12
query18	0.24	0.23	0.21
query19	2.02	1.93	1.92
query20	0.01	0.01	0.01
query21	15.36	0.24	0.15
query22	4.86	0.08	0.06
query23	15.65	0.31	0.12
query24	2.36	0.78	0.89
query25	0.08	0.06	0.07
query26	0.17	0.16	0.16
query27	0.08	0.07	0.07
query28	4.44	1.20	0.98
query29	12.68	4.57	3.77
query30	0.30	0.17	0.15
query31	2.84	0.65	0.41
query32	3.25	0.59	0.49
query33	3.11	3.20	3.19
query34	15.94	5.30	4.69
query35	4.70	4.61	4.59
query36	0.72	0.54	0.51
query37	0.11	0.07	0.08
query38	0.08	0.05	0.04
query39	0.04	0.04	0.04
query40	0.20	0.16	0.14
query41	0.09	0.04	0.04
query42	0.04	0.04	0.03
query43	0.05	0.04	0.05
Total cold run time: 98.95 s
Total hot run time: 29.57 s

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 9.72% (14/144) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.71% (18045/34236)
Line Coverage 37.96% (163748/431398)
Region Coverage 32.33% (124852/386191)
Branch Coverage 33.69% (54587/162005)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 90.07% (127/141) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.40% (24022/33644)
Line Coverage 57.83% (249814/431981)
Region Coverage 52.93% (207275/391625)
Branch Coverage 54.57% (88991/163078)

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 76.47% (13/17) 🎉
Increment coverage report
Complete coverage report

@linrrzqqq

Copy link
Copy Markdown
Collaborator Author

run buildall

@doris-robot

Copy link
Copy Markdown
ClickBench: Total hot run time: 28.79 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0ab8fbc78ea338f6e31bf84ddb9b74febb63a34a, data reload: false

query1	0.06	0.06	0.05
query2	0.10	0.05	0.05
query3	0.26	0.08	0.09
query4	1.61	0.13	0.12
query5	0.30	0.28	0.27
query6	1.18	0.68	0.69
query7	0.03	0.03	0.03
query8	0.06	0.04	0.04
query9	0.64	0.58	0.55
query10	0.62	0.62	0.62
query11	0.19	0.13	0.13
query12	0.18	0.14	0.13
query13	0.64	0.62	0.62
query14	1.04	1.04	1.03
query15	0.89	0.86	0.87
query16	0.43	0.41	0.41
query17	1.17	1.12	1.21
query18	0.23	0.22	0.23
query19	2.02	1.87	1.91
query20	0.01	0.01	0.02
query21	15.35	0.21	0.14
query22	5.00	0.08	0.05
query23	15.68	0.30	0.12
query24	2.67	0.76	0.30
query25	0.10	0.07	0.06
query26	0.16	0.15	0.16
query27	0.08	0.06	0.06
query28	4.20	1.19	0.96
query29	12.61	4.53	3.70
query30	0.28	0.15	0.14
query31	2.82	0.68	0.42
query32	3.24	0.59	0.48
query33	3.15	3.16	3.18
query34	16.04	5.23	4.58
query35	4.61	4.66	4.64
query36	0.72	0.54	0.56
query37	0.11	0.07	0.08
query38	0.07	0.05	0.05
query39	0.04	0.04	0.03
query40	0.19	0.16	0.14
query41	0.10	0.04	0.03
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 98.98 s
Total hot run time: 28.79 s

@doris-robot

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 9.72% (14/144) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.69% (18056/34266)
Line Coverage 37.98% (164080/431986)
Region Coverage 32.34% (125166/387062)
Branch Coverage 33.73% (54726/162243)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 90.07% (127/141) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.47% (24077/33686)
Line Coverage 57.87% (250415/432736)
Region Coverage 52.82% (207412/392658)
Branch Coverage 54.65% (89273/163354)

@linrrzqqq

Copy link
Copy Markdown
Collaborator Author

run buildall

@doris-robot

Copy link
Copy Markdown
TPC-DS: Total hot run time: 189481 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8486a80731d5080fc2d9d882df740e70da5f908d, data reload: false

query1	1045	412	397	397
query2	6560	1723	1724	1723
query3	6759	225	222	222
query4	26627	23325	23023	23023
query5	4646	632	458	458
query6	353	251	240	240
query7	4645	501	310	310
query8	325	269	252	252
query9	8763	2606	2624	2606
query10	511	343	309	309
query11	15814	15285	15660	15285
query12	207	139	126	126
query13	1802	594	499	499
query14	12945	9498	9810	9498
query15	299	191	181	181
query16	8101	709	563	563
query17	1971	839	663	663
query18	2929	462	342	342
query19	287	215	192	192
query20	140	131	136	131
query21	215	148	131	131
query22	4534	4820	4571	4571
query23	35070	33897	33488	33488
query24	8161	2461	2521	2461
query25	585	525	487	487
query26	1230	272	159	159
query27	2705	500	381	381
query28	4294	2173	2173	2173
query29	775	605	474	474
query30	295	219	187	187
query31	908	801	737	737
query32	80	70	62	62
query33	590	367	325	325
query34	771	866	506	506
query35	784	836	763	763
query36	960	992	911	911
query37	120	106	82	82
query38	3569	3537	3417	3417
query39	1463	1419	1402	1402
query40	214	129	113	113
query41	57	57	55	55
query42	118	112	107	107
query43	464	486	459	459
query44	1212	763	734	734
query45	184	175	174	174
query46	880	1077	642	642
query47	1754	1791	1676	1676
query48	389	418	316	316
query49	759	494	414	414
query50	634	681	403	403
query51	3922	3922	3887	3887
query52	111	108	100	100
query53	230	267	196	196
query54	304	290	273	273
query55	83	81	83	81
query56	326	308	315	308
query57	1166	1205	1120	1120
query58	287	273	262	262
query59	2542	2714	2612	2612
query60	342	363	319	319
query61	166	151	153	151
query62	813	756	644	644
query63	222	201	190	190
query64	4427	1161	845	845
query65	4018	3968	3932	3932
query66	1086	420	330	330
query67	15495	15035	15000	15000
query68	9225	890	591	591
query69	471	318	288	288
query70	1350	1313	1236	1236
query71	474	327	321	321
query72	5836	4912	4921	4912
query73	557	580	360	360
query74	8872	9008	8719	8719
query75	4296	3342	2790	2790
query76	3834	1216	727	727
query77	926	391	319	319
query78	9690	9832	8934	8934
query79	2004	824	605	605
query80	681	576	511	511
query81	486	260	224	224
query82	435	164	132	132
query83	308	263	248	248
query84	295	123	102	102
query85	896	480	430	430
query86	329	314	296	296
query87	3756	3766	3691	3691
query88	3070	2290	2270	2270
query89	397	330	291	291
query90	1970	219	217	217
query91	167	163	134	134
query92	87	69	70	69
query93	1166	958	647	647
query94	697	431	319	319
query95	411	319	317	317
query96	498	567	282	282
query97	2952	2988	2894	2894
query98	240	231	217	217
query99	1443	1391	1308	1308
Total cold run time: 281862 ms
Total hot run time: 189481 ms

@doris-robot

Copy link
Copy Markdown
ClickBench: Total hot run time: 27.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8486a80731d5080fc2d9d882df740e70da5f908d, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.05
query3	0.26	0.08	0.08
query4	1.61	0.12	0.11
query5	0.28	0.25	0.26
query6	1.16	0.65	0.64
query7	0.03	0.03	0.03
query8	0.06	0.04	0.05
query9	0.59	0.55	0.51
query10	0.60	0.58	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.62	0.61	0.61
query14	1.02	1.00	0.99
query15	0.85	0.84	0.84
query16	0.41	0.38	0.40
query17	1.06	1.05	1.02
query18	0.22	0.20	0.20
query19	1.90	1.80	1.83
query20	0.02	0.02	0.01
query21	15.45	0.18	0.14
query22	5.11	0.06	0.05
query23	15.64	0.26	0.10
query24	2.69	0.45	0.70
query25	0.07	0.06	0.06
query26	0.14	0.12	0.12
query27	0.06	0.06	0.06
query28	4.30	1.14	0.93
query29	12.59	3.91	3.20
query30	0.28	0.13	0.12
query31	2.81	0.59	0.39
query32	3.23	0.55	0.47
query33	3.01	3.06	3.13
query34	15.87	5.15	4.53
query35	4.54	4.59	4.59
query36	0.68	0.51	0.48
query37	0.10	0.06	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 98.16 s
Total hot run time: 27.5 s

@doris-robot

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 9.46% (14/148) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.68% (18050/34261)
Line Coverage 37.98% (164063/431996)
Region Coverage 32.32% (125067/386962)
Branch Coverage 33.73% (54723/162232)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 7.59% (11/145) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.25% (19281/33681)
Line Coverage 42.60% (184337/432746)
Region Coverage 37.09% (145583/392558)
Branch Coverage 38.14% (62307/163343)

1 similar comment
@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 7.59% (11/145) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.25% (19281/33681)
Line Coverage 42.60% (184337/432746)
Region Coverage 37.09% (145583/392558)
Branch Coverage 38.14% (62307/163343)

@linrrzqqq

Copy link
Copy Markdown
Collaborator Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 5.88% (1/17) 🎉
Increment coverage report
Complete coverage report

@doris-robot

Copy link
Copy Markdown
TPC-DS: Total hot run time: 186913 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a49bcfe67d4a00dd59be8eace0dad7155d3c8528, data reload: false

query1	1083	420	397	397
query2	6550	1734	1763	1734
query3	6757	226	229	226
query4	26372	23790	23324	23324
query5	4664	636	449	449
query6	326	240	235	235
query7	4653	491	301	301
query8	304	246	248	246
query9	8707	2590	2602	2590
query10	535	334	286	286
query11	15941	15112	14870	14870
query12	191	120	112	112
query13	1702	558	422	422
query14	11186	9190	9206	9190
query15	199	189	169	169
query16	7518	677	502	502
query17	1227	755	627	627
query18	2021	422	330	330
query19	236	237	201	201
query20	140	130	134	130
query21	223	140	129	129
query22	4648	4661	4801	4661
query23	34901	33779	33807	33779
query24	8779	2502	2499	2499
query25	630	549	503	503
query26	1808	286	170	170
query27	3423	518	358	358
query28	4524	2258	2229	2229
query29	851	657	497	497
query30	322	247	205	205
query31	949	827	764	764
query32	86	67	81	67
query33	603	391	349	349
query34	834	867	525	525
query35	852	874	826	826
query36	1059	1070	961	961
query37	139	113	85	85
query38	3646	3613	3485	3485
query39	1502	1392	1406	1392
query40	220	126	117	117
query41	60	57	58	57
query42	118	108	114	108
query43	485	498	478	478
query44	1236	726	729	726
query45	178	181	172	172
query46	886	991	639	639
query47	1761	1809	1733	1733
query48	400	429	333	333
query49	752	490	412	412
query50	666	689	407	407
query51	3823	3836	3919	3836
query52	113	108	96	96
query53	251	264	193	193
query54	316	320	293	293
query55	94	94	83	83
query56	359	352	326	326
query57	1191	1197	1108	1108
query58	301	289	283	283
query59	2586	2898	2508	2508
query60	376	360	337	337
query61	190	187	174	174
query62	802	708	680	680
query63	226	186	189	186
query64	4496	1179	869	869
query65	4066	3932	3938	3932
query66	1108	436	324	324
query67	15503	14916	15044	14916
query68	5955	886	596	596
query69	485	330	284	284
query70	1455	1265	1236	1236
query71	452	342	313	313
query72	5814	2570	5512	2570
query73	732	767	360	360
query74	8978	9171	8642	8642
query75	3307	3267	2841	2841
query76	3300	1140	720	720
query77	496	397	311	311
query78	9444	9725	8805	8805
query79	1645	887	609	609
query80	701	573	495	495
query81	526	267	238	238
query82	237	161	134	134
query83	263	266	255	255
query84	260	123	89	89
query85	914	506	443	443
query86	369	301	305	301
query87	3735	3728	3623	3623
query88	2860	2262	2256	2256
query89	383	332	291	291
query90	1867	217	203	203
query91	158	162	148	148
query92	72	67	63	63
query93	1188	989	644	644
query94	673	449	327	327
query95	406	321	319	319
query96	485	560	282	282
query97	2886	2955	2866	2866
query98	235	215	212	212
query99	1334	1413	1286	1286
Total cold run time: 274269 ms
Total hot run time: 186913 ms

@doris-robot

Copy link
Copy Markdown
ClickBench: Total hot run time: 27.66 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a49bcfe67d4a00dd59be8eace0dad7155d3c8528, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.06	0.05
query3	0.26	0.09	0.08
query4	1.61	0.12	0.11
query5	0.27	0.26	0.25
query6	1.18	0.64	0.63
query7	0.03	0.03	0.02
query8	0.06	0.04	0.05
query9	0.62	0.53	0.53
query10	0.57	0.57	0.57
query11	0.16	0.11	0.12
query12	0.15	0.12	0.11
query13	0.61	0.61	0.61
query14	1.01	1.01	1.01
query15	0.84	0.82	0.83
query16	0.39	0.39	0.43
query17	1.03	1.06	1.03
query18	0.21	0.19	0.19
query19	1.91	1.85	1.78
query20	0.01	0.01	0.01
query21	15.47	0.19	0.13
query22	5.13	0.07	0.05
query23	15.68	0.25	0.11
query24	2.67	0.60	0.70
query25	0.08	0.06	0.06
query26	0.15	0.13	0.13
query27	0.06	0.06	0.05
query28	4.57	1.12	0.95
query29	12.60	3.95	3.27
query30	0.29	0.13	0.13
query31	2.82	0.60	0.38
query32	3.22	0.55	0.48
query33	3.08	3.03	3.00
query34	15.71	5.21	4.54
query35	4.55	4.60	4.55
query36	0.70	0.51	0.49
query37	0.10	0.07	0.08
query38	0.06	0.04	0.03
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.03	0.04
query42	0.04	0.04	0.02
query43	0.05	0.04	0.03
Total cold run time: 98.39 s
Total hot run time: 27.66 s

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 10.64% (15/141) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.72% (18086/34305)
Line Coverage 38.04% (164490/432428)
Region Coverage 32.36% (125363/387365)
Branch Coverage 33.79% (54898/162446)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 93.48% (129/138) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.21% (24014/33724)
Line Coverage 57.61% (249559/433171)
Region Coverage 52.61% (206746/392959)
Branch Coverage 54.41% (88997/163555)

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 88.24% (15/17) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 14.58% (7/48) 🎉
Increment coverage report
Complete coverage report

@linrrzqqq

Copy link
Copy Markdown
Collaborator Author

run p0

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 83.33% (40/48) 🎉
Increment coverage report
Complete coverage report

| CASE value=expression whenClause+ (ELSE elseExpression=expression)? END #simpleCase
| name=CAST LEFT_PAREN expression AS castDataType RIGHT_PAREN #cast
| name=TRY_CAST LEFT_PAREN expression AS castDataType RIGHT_PAREN #tryCast
| DEFAULT LEFT_PAREN expression RIGHT_PAREN #defaultValue

@924060929 924060929 Jan 22, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be DEFAULT LEFT_PAREN qualifiedName RIGHT_PAREN because you only support get default value from a column

@linrrzqqq

Copy link
Copy Markdown
Collaborator Author

run buildall

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Jan 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@doris-robot

Copy link
Copy Markdown
TPC-H: Total hot run time: 30813 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 250ef6b99c1678f3b80b565c8d601f423dba7d9a, data reload: false

------ Round 1 ----------------------------------
q1	17711	4773	4569	4569
q2	2005	321	202	202
q3	10271	1309	726	726
q4	10217	863	311	311
q5	7553	2052	1841	1841
q6	183	169	141	141
q7	860	712	584	584
q8	9260	1384	1072	1072
q9	4907	4605	4667	4605
q10	6801	1643	1259	1259
q11	538	301	284	284
q12	341	377	225	225
q13	17785	3826	3097	3097
q14	241	245	216	216
q15	608	521	528	521
q16	633	624	577	577
q17	652	774	529	529
q18	6521	6342	6321	6321
q19	1100	980	626	626
q20	402	353	238	238
q21	2680	1927	1962	1927
q22	1065	985	942	942
Total cold run time: 102334 ms
Total hot run time: 30813 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4715	4737	4672	4672
q2	307	398	315	315
q3	2138	2658	2240	2240
q4	1323	1769	1280	1280
q5	4112	3994	4107	3994
q6	217	175	133	133
q7	1939	1944	1762	1762
q8	2899	2546	2646	2546
q9	7350	7257	7091	7091
q10	2470	2730	2268	2268
q11	574	500	460	460
q12	727	733	646	646
q13	3640	4103	3429	3429
q14	288	313	275	275
q15	555	513	508	508
q16	643	680	662	662
q17	1173	1346	1406	1346
q18	8204	7585	7773	7585
q19	939	827	867	827
q20	1967	2124	1970	1970
q21	4678	4249	4100	4100
q22	1071	997	991	991
Total cold run time: 51929 ms
Total hot run time: 49100 ms

@doris-robot

Copy link
Copy Markdown
TPC-DS: Total hot run time: 172507 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 250ef6b99c1678f3b80b565c8d601f423dba7d9a, data reload: false

query5	4404	656	487	487
query6	310	217	196	196
query7	4217	450	255	255
query8	337	243	230	230
query9	8701	2832	2856	2832
query10	438	309	284	284
query11	15193	15226	14832	14832
query12	185	121	117	117
query13	1248	464	378	378
query14	6232	2963	2752	2752
query14_1	2672	2775	2638	2638
query15	198	188	170	170
query16	964	485	456	456
query17	1088	686	574	574
query18	2450	438	333	333
query19	204	182	157	157
query20	123	117	113	113
query21	215	139	124	124
query22	4169	4217	4052	4052
query23	15978	15533	15453	15453
query23_1	15638	15450	15600	15450
query24	7134	1548	1164	1164
query24_1	1157	1174	1170	1170
query25	532	460	413	413
query26	1243	272	154	154
query27	2767	464	276	276
query28	4507	2150	2178	2150
query29	803	541	447	447
query30	309	241	201	201
query31	805	636	564	564
query32	88	76	78	76
query33	538	376	312	312
query34	909	881	530	530
query35	723	756	675	675
query36	823	877	817	817
query37	138	102	89	89
query38	2697	2679	2720	2679
query39	782	733	761	733
query39_1	711	717	703	703
query40	221	137	122	122
query41	74	67	66	66
query42	99	95	94	94
query43	435	445	406	406
query44	1311	745	746	745
query45	188	185	178	178
query46	824	947	607	607
query47	1401	1471	1380	1380
query48	303	319	241	241
query49	597	432	340	340
query50	690	275	199	199
query51	3805	3746	3692	3692
query52	90	90	84	84
query53	206	210	165	165
query54	272	259	244	244
query55	79	84	75	75
query56	302	294	287	287
query57	1092	1044	957	957
query58	270	262	270	262
query59	2039	2074	2037	2037
query60	330	332	313	313
query61	155	148	146	146
query62	390	361	314	314
query63	191	155	166	155
query64	4946	1181	843	843
query65	3837	3647	3768	3647
query66	1455	426	318	318
query67	15531	15546	15418	15418
query68	2497	1061	731	731
query69	385	311	270	270
query70	997	912	954	912
query71	306	292	276	276
query72	5295	3174	3229	3174
query73	601	727	326	326
query74	8762	8733	8606	8606
query75	2314	2294	1917	1917
query76	2266	1046	641	641
query77	361	395	296	296
query78	9647	9797	9115	9115
query79	2536	889	602	602
query80	1745	530	435	435
query81	541	262	235	235
query82	1016	152	118	118
query83	324	262	244	244
query84	256	117	101	101
query85	872	462	418	418
query86	412	293	288	288
query87	2875	2866	2793	2793
query88	3526	2587	2570	2570
query89	312	248	240	240
query90	2011	178	166	166
query91	167	157	132	132
query92	76	70	68	68
query93	1226	1022	638	638
query94	638	327	279	279
query95	594	336	379	336
query96	640	512	231	231
query97	2381	2387	2324	2324
query98	232	200	197	197
query99	607	565	478	478
Total cold run time: 248054 ms
Total hot run time: 172507 ms

@doris-robot

Copy link
Copy Markdown
ClickBench: Total hot run time: 26.69 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 250ef6b99c1678f3b80b565c8d601f423dba7d9a, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.04	0.05
query3	0.26	0.08	0.08
query4	1.61	0.11	0.11
query5	0.27	0.25	0.25
query6	1.14	0.64	0.66
query7	0.04	0.03	0.02
query8	0.05	0.04	0.04
query9	0.57	0.51	0.50
query10	0.56	0.54	0.55
query11	0.15	0.10	0.11
query12	0.15	0.11	0.11
query13	0.61	0.58	0.60
query14	0.97	0.96	0.93
query15	0.80	0.78	0.79
query16	0.41	0.39	0.39
query17	0.98	1.06	1.01
query18	0.24	0.22	0.22
query19	1.88	1.77	1.82
query20	0.02	0.01	0.01
query21	15.43	0.28	0.13
query22	5.34	0.05	0.04
query23	16.17	0.29	0.10
query24	0.92	0.23	0.60
query25	0.09	0.07	0.10
query26	0.15	0.13	0.14
query27	0.09	0.08	0.05
query28	4.17	1.09	0.89
query29	12.53	3.94	3.14
query30	0.27	0.14	0.11
query31	2.81	0.64	0.39
query32	3.25	0.55	0.46
query33	3.04	3.01	3.00
query34	15.80	5.09	4.47
query35	4.50	4.45	4.46
query36	0.65	0.50	0.50
query37	0.10	0.07	0.06
query38	0.08	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.14	0.13
query41	0.10	0.03	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 96.65 s
Total hot run time: 26.69 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 13.21% (7/53) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 83.02% (44/53) 🎉
Increment coverage report
Complete coverage report

@zclllyybb
zclllyybb merged commit 8c5b9ae into apache:master Jan 23, 2026
30 checks passed
github-actions Bot pushed a commit that referenced this pull request Jan 23, 2026
Return the default value of column

```sql
CREATE TABLE test_default_scalar (
    c_id INT,
    c_bool                   BOOLEAN                               NULL     DEFAULT 1,
    c_tinyint                TINYINT                               NULL     DEFAULT 7,
    c_smallint               SMALLINT                              NULL     DEFAULT 32000,
    c_int                    INT                                   NULL     DEFAULT 2147483647,
    c_bigint                 BIGINT                                NULL     DEFAULT 9223372036854775807,
    c_largeint               LARGEINT                              NULL     DEFAULT '170141183460469231731687303715884105727',
    c_float                  FLOAT                                 NULL     DEFAULT 3.125,
    c_double                 DOUBLE                                NULL     DEFAULT 2.718281828,
    c_decimal                DECIMAL(27, 9)                        NULL     DEFAULT '123456789.123456789',
    c_decimal_compact        DECIMAL(18, 4)                        NULL     DEFAULT '99999.1234',
    c_pi                     DOUBLE                                NULL     DEFAULT PI,
    c_e                      DOUBLE                                NULL     DEFAULT E,
    c_char                   CHAR(8)                               NULL     DEFAULT 'charDemo',
    c_varchar                VARCHAR(32)                           NULL     DEFAULT '',
    c_string                 STRING                                NULL     DEFAULT 'plain string',
    c_datetime               DATETIME                              NULL     DEFAULT '2025-10-25 11:22:33',
    c_date                   DATE                                  NULL     DEFAULT '2025-10-31',
    c_json                   JSON                                  NULL,
    c_ipv4                   IPV4                                  NULL     DEFAULT '192.168.1.1',
    c_ipv6                   IPV6                                  NULL     DEFAULT '2001:db8::1',
    c_array_int              ARRAY<INT>                            NULL     DEFAULT '[]',
    c_array_string           ARRAY<STRING>                         NULL,
    c_map_str_int            MAP<STRING, INT>                      NULL,
    c_struct                 STRUCT<f1:INT,f2:STRING,f3:BOOLEAN>   NULL,
    c_variant                VARIANT                               NULL
) PROPERTIES ( 'replication_num' = '1');

SELECT
    default(c_id)                   AS default_id,
    default(c_bool)                 AS default_c_bool,
    default(c_tinyint)              AS default_c_tinyint,
    default(c_smallint)             AS default_c_smallint,
    default(c_int)                  AS default_c_int,
    default(c_bigint)               AS default_c_bigint,
    default(c_largeint)             AS default_c_largeint,
    default(c_float)                AS default_c_float,
    default(c_double)               AS default_c_double,
    default(c_decimal)              AS default_c_decimal,
    default(c_decimal_compact)      AS default_c_decimal_compact,
    default(c_char)                 AS default_c_char,
    default(c_varchar)              AS default_c_varchar,
    default(c_string)               AS default_c_string,
    default(c_datetime)             AS default_datetime,
    default(c_date)                 AS defaut_time,
    default(c_json)                 AS default_c_json,
    default(c_ipv4)                 AS default_c_ipv4,
    default(c_ipv6)                 AS default_c_ipv6,
    default(c_array_int)            AS default_c_array_int,
    default(c_array_string)         AS default_c_array_string,
    default(c_map_str_int)          AS default_c_map_str_int,
    default(c_struct)               AS default_c_struct,
    default(c_variant)              AS default_c_variant
FROM test_default_scalar
LIMIT 1;
```
```text
+------------+----------------+-------------------+--------------------+---------------+---------------------+-----------------------------------------+-----------------+------------------+---------------------+---------------------------+----------------+-------------------+------------------+---------------------+-------------+----------------+----------------+----------------+---------------------+------------------------+-----------------------+------------------+-------------------+
| default_id | default_c_bool | default_c_tinyint | default_c_smallint | default_c_int | default_c_bigint    | default_c_largeint                      | default_c_float | default_c_double | default_c_decimal   | default_c_decimal_compact | default_c_char | default_c_varchar | default_c_string | default_datetime    | defaut_time | default_c_json | default_c_ipv4 | default_c_ipv6 | default_c_array_int | default_c_array_string | default_c_map_str_int | default_c_struct | default_c_variant |
+------------+----------------+-------------------+--------------------+---------------+---------------------+-----------------------------------------+-----------------+------------------+---------------------+---------------------------+----------------+-------------------+------------------+---------------------+-------------+----------------+----------------+----------------+---------------------+------------------------+-----------------------+------------------+-------------------+
|       NULL |              1 |                 7 |              32000 |    2147483647 | 9223372036854775807 | 170141183460469231731687303715884105727 |           3.125 |      2.718281828 | 123456789.123456789 |                99999.1234 | charDemo       |                   | plain string     | 2025-10-25 11:22:33 | 2025-10-31  | NULL           | 192.168.1.1    | 2001:db8::1    | []                  | NULL                   | NULL                  | NULL             | NULL              |
+------------+----------------+-------------------+--------------------+---------------+---------------------+-----------------------------------------+-----------------+------------------+---------------------+---------------------------+----------------+-------------------+------------------+---------------------+-------------+----------------+----------------+----------------+---------------------+------------------------+-----------------------+------------------+-------------------+
```
@linrrzqqq
linrrzqqq deleted the default branch January 26, 2026 01:55
morningman added a commit to morningman/doris that referenced this pull request Oct 23, 2026
…hema

### What problem does this PR solve?

Issue Number: None

Related PR: apache#66538, apache#57531, apache#65851

Problem Summary: An Iceberg write-default exists only in the writer schema
that a write statement pins (IcebergWriteSchemaContext); the columns of the
read schema carry no default. Master resolves DEFAULT(column) against that
writer schema only in UPDATE / MERGE SET and in MERGE INSERT VALUES, and
everywhere else falls back to the read column. On a v3 table whose column d
has write-default 42, INSERT ... VALUES (DEFAULT(d)) and INSERT ... SELECT
DEFAULT(d) without FROM fail with "Unknown column", INSERT ... SELECT
DEFAULT(d) FROM t writes NULL, UPDATE ... WHERE d = DEFAULT(d) and MERGE
... ON ... d = DEFAULT(d) compare d with NULL and silently match no row, and
SELECT DEFAULT(d) FROM t returns NULL. branch-4.1 and branch-4.2 (apache#66538)
resolve all of these against the pinned write target, by name against the
target in INSERT ... SELECT, and reject DEFAULT(column) on an Iceberg column
in a statement that does not write the table.

This change adds the connector capability SUPPORTS_WRITE_ONLY_DEFAULTS,
declared by the Iceberg connector and inherited by iceberg-on-HMS tables.
For a table that declares it, InsertUtils resolves DEFAULT(column) in INSERT
VALUES and in the select lists of INSERT ... SELECT by name against the
pinned writer schema, and RewriteDefaultExpression takes the default of a
DEFAULT(column) bound to the table from the pinned writer column with the
same field id, which covers the UPDATE and MERGE conditions, and rejects it
when the statement pinned no writer schema for the table. Hive, Paimon and
OLAP tables keep the current behavior. The new enum constant changes the
connector plugin API surface, so the plugin API version goes from 20.0 to
21.0.

### Release note

Fix: On Iceberg tables, DEFAULT(column) in INSERT VALUES, INSERT ... SELECT
and the WHERE / ON conditions of UPDATE and MERGE now resolves to the
column's write-default. Before, it failed, wrote NULL, or matched no row.
SELECT DEFAULT(column) on an Iceberg table now fails instead of returning
NULL.

### Check List (For Author)

- Test: Regression test / Unit Test
    - test_iceberg_initial_defaults: writes rows 20-22 with
      DEFAULT(default_int) in VALUES and in INSERT ... SELECT with and
      without FROM, then updates them with UPDATE ... WHERE default_int =
      DEFAULT(default_int) and MERGE ... ON ... = DEFAULT(...), for Parquet
      and ORC; SELECT DEFAULT(default_int) must fail. Passes locally; with
      master's FE it fails at the first INSERT with "Unknown column
      'default_int'"
    - InsertUtilsTest: VALUES and INSERT ... SELECT (from a staging relation
      and without FROM) resolve against the pinned writer schema; unknown
      column fails; a table without the capability is unchanged
    - RewriteDefaultExpressionTest: pinned writer column wins, no pinned
      schema fails, a table without the capability keeps the read default
    - IcebergConnectorTest, HiveConnectorMetadataSiblingDelegationTest,
      ConnectorPluginSurfaceTest
- Behavior changed: Yes. See the release note.
- Does this need documentation: No

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
morningman added a commit to morningman/doris that referenced this pull request Oct 23, 2026
…hema

### What problem does this PR solve?

Issue Number: None

Related PR: apache#66538, apache#57531, apache#65851

Problem Summary: An Iceberg write-default exists only in the writer schema
that a write statement pins (IcebergWriteSchemaContext); the columns of the
read schema carry no default. Master resolves DEFAULT(column) against that
writer schema only in UPDATE / MERGE SET and in MERGE INSERT VALUES, and
everywhere else falls back to the read column. On a v3 table whose column d
has write-default 42, INSERT ... VALUES (DEFAULT(d)) and INSERT ... SELECT
DEFAULT(d) without FROM fail with "Unknown column", INSERT ... SELECT
DEFAULT(d) FROM t writes NULL, UPDATE ... WHERE d = DEFAULT(d) and MERGE
... ON ... d = DEFAULT(d) compare d with NULL and silently match no row, and
SELECT DEFAULT(d) FROM t returns NULL. branch-4.1 and branch-4.2 (apache#66538)
resolve all of these against the pinned write target, by name against the
target in INSERT ... SELECT, and reject DEFAULT(column) on an Iceberg column
in a statement that does not write the table.

This change adds the connector capability SUPPORTS_WRITE_ONLY_DEFAULTS,
declared by the Iceberg connector and inherited by iceberg-on-HMS tables.
For a table that declares it, InsertUtils resolves DEFAULT(column) in INSERT
VALUES and in the select lists of INSERT ... SELECT by name against the
pinned writer schema, and RewriteDefaultExpression takes the default of a
DEFAULT(column) bound to the table from the pinned writer column with the
same field id, which covers the UPDATE and MERGE conditions, and rejects it
when the statement pinned no writer schema for the table. Hive, Paimon and
OLAP tables keep the current behavior. The new enum constant changes the
connector plugin API surface, so the plugin API version goes from 20.0 to
21.0.

### Release note

Fix: On Iceberg tables, DEFAULT(column) in INSERT VALUES, INSERT ... SELECT
and the WHERE / ON conditions of UPDATE and MERGE now resolves to the
column's write-default. Before, it failed, wrote NULL, or matched no row.
SELECT DEFAULT(column) on an Iceberg table now fails instead of returning
NULL.

### Check List (For Author)

- Test: Regression test / Unit Test
    - test_iceberg_initial_defaults: writes rows 20-22 with
      DEFAULT(default_int) in VALUES and in INSERT ... SELECT with and
      without FROM, then updates them with UPDATE ... WHERE default_int =
      DEFAULT(default_int) and MERGE ... ON ... = DEFAULT(...), for Parquet
      and ORC; SELECT DEFAULT(default_int) must fail. Passes locally; with
      master's FE it fails at the first INSERT with "Unknown column
      'default_int'"
    - InsertUtilsTest: VALUES and INSERT ... SELECT (from a staging relation
      and without FROM) resolve against the pinned writer schema; unknown
      column fails; a table without the capability is unchanged
    - RewriteDefaultExpressionTest: pinned writer column wins, no pinned
      schema fails, a table without the capability keeps the read default
    - IcebergConnectorTest, HiveConnectorMetadataSiblingDelegationTest,
      ConnectorPluginSurfaceTest
- Behavior changed: Yes. See the release note.
- Does this need documentation: No

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
morningman added a commit to morningman/doris that referenced this pull request Oct 25, 2026
…hema

### What problem does this PR solve?

Issue Number: None

Related PR: apache#66538, apache#57531, apache#65851

Problem Summary: An Iceberg write-default exists only in the writer schema
that a write statement pins (IcebergWriteSchemaContext); the columns of the
read schema carry no default. Master resolves DEFAULT(column) against that
writer schema only in UPDATE / MERGE SET and in MERGE INSERT VALUES, and
everywhere else falls back to the read column. On a v3 table whose column d
has write-default 42, INSERT ... VALUES (DEFAULT(d)) and INSERT ... SELECT
DEFAULT(d) without FROM fail with "Unknown column", INSERT ... SELECT
DEFAULT(d) FROM t writes NULL, UPDATE ... WHERE d = DEFAULT(d) and MERGE
... ON ... d = DEFAULT(d) compare d with NULL and silently match no row, and
SELECT DEFAULT(d) FROM t returns NULL. branch-4.1 and branch-4.2 (apache#66538)
resolve all of these against the pinned write target, by name against the
target in INSERT ... SELECT, and reject DEFAULT(column) on an Iceberg column
in a statement that does not write the table.

This change adds the connector capability SUPPORTS_WRITE_ONLY_DEFAULTS,
declared by the Iceberg connector and inherited by iceberg-on-HMS tables.
For a table that declares it, InsertUtils resolves DEFAULT(column) in INSERT
VALUES and in the select lists of INSERT ... SELECT by name against the
pinned writer schema, and RewriteDefaultExpression takes the default of a
DEFAULT(column) bound to the table from the pinned writer column with the
same field id, which covers the UPDATE and MERGE conditions, and rejects it
when the statement pinned no writer schema for the table. Hive, Paimon and
OLAP tables keep the current behavior. The new enum constant changes the
connector plugin API surface, so the plugin API version goes from 20.0 to
21.0.

### Release note

Fix: On Iceberg tables, DEFAULT(column) in INSERT VALUES, INSERT ... SELECT
and the WHERE / ON conditions of UPDATE and MERGE now resolves to the
column's write-default. Before, it failed, wrote NULL, or matched no row.
SELECT DEFAULT(column) on an Iceberg table now fails instead of returning
NULL.

### Check List (For Author)

- Test: Regression test / Unit Test
    - test_iceberg_initial_defaults: writes rows 20-22 with
      DEFAULT(default_int) in VALUES and in INSERT ... SELECT with and
      without FROM, then updates them with UPDATE ... WHERE default_int =
      DEFAULT(default_int) and MERGE ... ON ... = DEFAULT(...), for Parquet
      and ORC; SELECT DEFAULT(default_int) must fail. Passes locally; with
      master's FE it fails at the first INSERT with "Unknown column
      'default_int'"
    - InsertUtilsTest: VALUES and INSERT ... SELECT (from a staging relation
      and without FROM) resolve against the pinned writer schema; unknown
      column fails; a table without the capability is unchanged
    - RewriteDefaultExpressionTest: pinned writer column wins, no pinned
      schema fails, a table without the capability keeps the read default
    - IcebergConnectorTest, HiveConnectorMetadataSiblingDelegationTest,
      ConnectorPluginSurfaceTest
- Behavior changed: Yes. See the release note.
- Does this need documentation: No

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
morningman added a commit to morningman/doris that referenced this pull request Oct 26, 2026
…hema

### What problem does this PR solve?

Issue Number: None

Related PR: apache#66538, apache#57531, apache#65851

Problem Summary: An Iceberg write-default exists only in the writer schema
that a write statement pins (IcebergWriteSchemaContext); the columns of the
read schema carry no default. Master resolves DEFAULT(column) against that
writer schema only in UPDATE / MERGE SET and in MERGE INSERT VALUES, and
everywhere else falls back to the read column. On a v3 table whose column d
has write-default 42, INSERT ... VALUES (DEFAULT(d)) and INSERT ... SELECT
DEFAULT(d) without FROM fail with "Unknown column", INSERT ... SELECT
DEFAULT(d) FROM t writes NULL, UPDATE ... WHERE d = DEFAULT(d) and MERGE
... ON ... d = DEFAULT(d) compare d with NULL and silently match no row, and
SELECT DEFAULT(d) FROM t returns NULL. branch-4.1 and branch-4.2 (apache#66538)
resolve all of these against the pinned write target, by name against the
target in INSERT ... SELECT, and reject DEFAULT(column) on an Iceberg column
in a statement that does not write the table.

This change adds the connector capability SUPPORTS_WRITE_ONLY_DEFAULTS,
declared by the Iceberg connector and inherited by iceberg-on-HMS tables.
For a table that declares it, InsertUtils resolves DEFAULT(column) in INSERT
VALUES and in the select lists of INSERT ... SELECT by name against the
pinned writer schema, and RewriteDefaultExpression takes the default of a
DEFAULT(column) bound to the table from the pinned writer column with the
same field id, which covers the UPDATE and MERGE conditions, and rejects it
when the statement pinned no writer schema for the table. Hive, Paimon and
OLAP tables keep the current behavior. The new enum constant changes the
connector plugin API surface, so the plugin API version goes from 21.0 to
22.0.

### Release note

Fix: On Iceberg tables, DEFAULT(column) in INSERT VALUES, INSERT ... SELECT
and the WHERE / ON conditions of UPDATE and MERGE now resolves to the
column's write-default. Before, it failed, wrote NULL, or matched no row.
SELECT DEFAULT(column) on an Iceberg table now fails instead of returning
NULL.

### Check List (For Author)

- Test: Regression test / Unit Test
    - test_iceberg_initial_defaults: writes rows 20-22 with
      DEFAULT(default_int) in VALUES and in INSERT ... SELECT with and
      without FROM, then updates them with UPDATE ... WHERE default_int =
      DEFAULT(default_int) and MERGE ... ON ... = DEFAULT(...), for Parquet
      and ORC; SELECT DEFAULT(default_int) must fail. Passes locally; with
      master's FE it fails at the first INSERT with "Unknown column
      'default_int'"
    - InsertUtilsTest: VALUES and INSERT ... SELECT (from a staging relation
      and without FROM) resolve against the pinned writer schema; unknown
      column fails; a table without the capability is unchanged
    - RewriteDefaultExpressionTest: pinned writer column wins, no pinned
      schema fails, a table without the capability keeps the read default
    - IcebergConnectorTest, HiveConnectorMetadataSiblingDelegationTest,
      ConnectorPluginSurfaceTest
- Behavior changed: Yes. See the release note.
- Does this need documentation: No

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
morningman added a commit to morningman/doris that referenced this pull request Oct 27, 2026
…hema

### What problem does this PR solve?

Issue Number: None

Related PR: apache#66538, apache#57531, apache#65851

Problem Summary: An Iceberg write-default exists only in the writer schema
that a write statement pins (IcebergWriteSchemaContext); the columns of the
read schema carry no default. Master resolves DEFAULT(column) against that
writer schema only in UPDATE / MERGE SET and in MERGE INSERT VALUES, and
everywhere else falls back to the read column. On a v3 table whose column d
has write-default 42, INSERT ... VALUES (DEFAULT(d)) and INSERT ... SELECT
DEFAULT(d) without FROM fail with "Unknown column", INSERT ... SELECT
DEFAULT(d) FROM t writes NULL, UPDATE ... WHERE d = DEFAULT(d) and MERGE
... ON ... d = DEFAULT(d) compare d with NULL and silently match no row, and
SELECT DEFAULT(d) FROM t returns NULL. branch-4.1 and branch-4.2 (apache#66538)
resolve all of these against the pinned write target, by name against the
target in INSERT ... SELECT, and reject DEFAULT(column) on an Iceberg column
in a statement that does not write the table.

This change adds the connector capability SUPPORTS_WRITE_ONLY_DEFAULTS,
declared by the Iceberg connector and inherited by iceberg-on-HMS tables.
For a table that declares it, InsertUtils resolves DEFAULT(column) in INSERT
VALUES and in the select lists of INSERT ... SELECT by name against the
pinned writer schema, and RewriteDefaultExpression takes the default of a
DEFAULT(column) bound to the table from the pinned writer column with the
same field id, which covers the UPDATE and MERGE conditions, and rejects it
when the statement pinned no writer schema for the table. Hive, Paimon and
OLAP tables keep the current behavior. The new enum constant changes the
connector plugin API surface, so the plugin API version goes from 24.0 to
25.0.

### Release note

Fix: On Iceberg tables, DEFAULT(column) in INSERT VALUES, INSERT ... SELECT
and the WHERE / ON conditions of UPDATE and MERGE now resolves to the
column's write-default. Before, it failed, wrote NULL, or matched no row.
SELECT DEFAULT(column) on an Iceberg table now fails instead of returning
NULL.

### Check List (For Author)

- Test: Regression test / Unit Test
    - test_iceberg_initial_defaults: writes rows 20-22 with
      DEFAULT(default_int) in VALUES and in INSERT ... SELECT with and
      without FROM, then updates them with UPDATE ... WHERE default_int =
      DEFAULT(default_int) and MERGE ... ON ... = DEFAULT(...), for Parquet
      and ORC; SELECT DEFAULT(default_int) must fail. Passes locally; with
      master's FE it fails at the first INSERT with "Unknown column
      'default_int'"
    - InsertUtilsTest: VALUES and INSERT ... SELECT (from a staging relation
      and without FROM) resolve against the pinned writer schema; unknown
      column fails; a table without the capability is unchanged
    - RewriteDefaultExpressionTest: pinned writer column wins, no pinned
      schema fails, a table without the capability keeps the read default
    - IcebergConnectorTest, HiveConnectorMetadataSiblingDelegationTest,
      ConnectorPluginSurfaceTest
- Behavior changed: Yes. See the release note.
- Does this need documentation: No

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
morningman added a commit to morningman/doris that referenced this pull request Oct 29, 2026
…hema

### What problem does this PR solve?

Issue Number: None

Related PR: apache#66538, apache#57531, apache#65851

Problem Summary: An Iceberg write-default exists only in the writer schema
that a write statement pins (IcebergWriteSchemaContext); the columns of the
read schema carry no default. Master resolves DEFAULT(column) against that
writer schema only in UPDATE / MERGE SET and in MERGE INSERT VALUES, and
everywhere else falls back to the read column. On a v3 table whose column d
has write-default 42, INSERT ... VALUES (DEFAULT(d)) and INSERT ... SELECT
DEFAULT(d) without FROM fail with "Unknown column", INSERT ... SELECT
DEFAULT(d) FROM t writes NULL, UPDATE ... WHERE d = DEFAULT(d) and MERGE
... ON ... d = DEFAULT(d) compare d with NULL and silently match no row, and
SELECT DEFAULT(d) FROM t returns NULL. branch-4.1 and branch-4.2 (apache#66538)
resolve all of these against the pinned write target, by name against the
target in INSERT ... SELECT, and reject DEFAULT(column) on an Iceberg column
in a statement that does not write the table.

This change adds the connector capability SUPPORTS_WRITE_ONLY_DEFAULTS,
declared by the Iceberg connector and inherited by iceberg-on-HMS tables.
For a table that declares it, InsertUtils resolves DEFAULT(column) in INSERT
VALUES and in the select lists of INSERT ... SELECT by name against the
pinned writer schema, and RewriteDefaultExpression takes the default of a
DEFAULT(column) bound to the table from the pinned writer column with the
same field id, which covers the UPDATE and MERGE conditions, and rejects it
when the statement pinned no writer schema for the table. Hive, Paimon and
OLAP tables keep the current behavior. The new enum constant changes the
connector plugin API surface, so the plugin API version goes from 25.0 to
26.0.

### Release note

Fix: On Iceberg tables, DEFAULT(column) in INSERT VALUES, INSERT ... SELECT
and the WHERE / ON conditions of UPDATE and MERGE now resolves to the
column's write-default. Before, it failed, wrote NULL, or matched no row.
SELECT DEFAULT(column) on an Iceberg table now fails instead of returning
NULL.

### Check List (For Author)

- Test: Regression test / Unit Test
    - test_iceberg_initial_defaults: writes rows 20-22 with
      DEFAULT(default_int) in VALUES and in INSERT ... SELECT with and
      without FROM, then updates them with UPDATE ... WHERE default_int =
      DEFAULT(default_int) and MERGE ... ON ... = DEFAULT(...), for Parquet
      and ORC; SELECT DEFAULT(default_int) must fail. Passes locally; with
      master's FE it fails at the first INSERT with "Unknown column
      'default_int'"
    - InsertUtilsTest: VALUES and INSERT ... SELECT (from a staging relation
      and without FROM) resolve against the pinned writer schema; unknown
      column fails; a table without the capability is unchanged
    - RewriteDefaultExpressionTest: pinned writer column wins, no pinned
      schema fails, a table without the capability keeps the read default
    - IcebergConnectorTest, HiveConnectorMetadataSiblingDelegationTest,
      ConnectorPluginSurfaceTest
- Behavior changed: Yes. See the release note.
- Does this need documentation: No

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
morningman added a commit to morningman/doris that referenced this pull request Oct 29, 2026
…hema

### What problem does this PR solve?

Issue Number: None

Related PR: apache#66538, apache#57531, apache#65851

Problem Summary: An Iceberg write-default exists only in the writer schema
that a write statement pins (IcebergWriteSchemaContext); the columns of the
read schema carry no default. Master resolves DEFAULT(column) against that
writer schema only in UPDATE / MERGE SET and in MERGE INSERT VALUES, and
everywhere else falls back to the read column. On a v3 table whose column d
has write-default 42, INSERT ... VALUES (DEFAULT(d)) and INSERT ... SELECT
DEFAULT(d) without FROM fail with "Unknown column", INSERT ... SELECT
DEFAULT(d) FROM t writes NULL, UPDATE ... WHERE d = DEFAULT(d) and MERGE
... ON ... d = DEFAULT(d) compare d with NULL and silently match no row, and
SELECT DEFAULT(d) FROM t returns NULL. branch-4.1 and branch-4.2 (apache#66538)
resolve all of these against the pinned write target, by name against the
target in INSERT ... SELECT, and reject DEFAULT(column) on an Iceberg column
in a statement that does not write the table.

This change adds the connector capability SUPPORTS_WRITE_ONLY_DEFAULTS,
declared by the Iceberg connector and inherited by iceberg-on-HMS tables.
For a table that declares it, InsertUtils resolves DEFAULT(column) in INSERT
VALUES and in the select lists of INSERT ... SELECT by name against the
pinned writer schema, and RewriteDefaultExpression takes the default of a
DEFAULT(column) bound to the table from the pinned writer column with the
same field id, which covers the UPDATE and MERGE conditions, and rejects it
when the statement pinned no writer schema for the table. Hive, Paimon and
OLAP tables keep the current behavior. The new enum constant changes the
connector plugin API surface, so the plugin API version goes from 25.0 to
26.0.

### Release note

Fix: On Iceberg tables, DEFAULT(column) in INSERT VALUES, INSERT ... SELECT
and the WHERE / ON conditions of UPDATE and MERGE now resolves to the
column's write-default. Before, it failed, wrote NULL, or matched no row.
SELECT DEFAULT(column) on an Iceberg table now fails instead of returning
NULL.

### Check List (For Author)

- Test: Regression test / Unit Test
    - test_iceberg_initial_defaults: writes rows 20-22 with
      DEFAULT(default_int) in VALUES and in INSERT ... SELECT with and
      without FROM, then updates them with UPDATE ... WHERE default_int =
      DEFAULT(default_int) and MERGE ... ON ... = DEFAULT(...), for Parquet
      and ORC; SELECT DEFAULT(default_int) must fail. Passes locally; with
      master's FE it fails at the first INSERT with "Unknown column
      'default_int'"
    - InsertUtilsTest: VALUES and INSERT ... SELECT (from a staging relation
      and without FROM) resolve against the pinned writer schema; unknown
      column fails; a table without the capability is unchanged
    - RewriteDefaultExpressionTest: pinned writer column wins, no pinned
      schema fails, a table without the capability keeps the read default
    - IcebergConnectorTest, HiveConnectorMetadataSiblingDelegationTest,
      ConnectorPluginSurfaceTest
- Behavior changed: Yes. See the release note.
- Does this need documentation: No

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
morningman added a commit to morningman/doris that referenced this pull request Oct 31, 2026
…hema

### What problem does this PR solve?

Issue Number: None

Related PR: apache#66538, apache#57531, apache#65851

Problem Summary: An Iceberg write-default exists only in the writer schema
that a write statement pins (IcebergWriteSchemaContext); the columns of the
read schema carry no default. Master resolves DEFAULT(column) against that
writer schema only in UPDATE / MERGE SET and in MERGE INSERT VALUES, and
everywhere else falls back to the read column. On a v3 table whose column d
has write-default 42, INSERT ... VALUES (DEFAULT(d)) and INSERT ... SELECT
DEFAULT(d) without FROM fail with "Unknown column", INSERT ... SELECT
DEFAULT(d) FROM t writes NULL, UPDATE ... WHERE d = DEFAULT(d) and MERGE
... ON ... d = DEFAULT(d) compare d with NULL and silently match no row, and
SELECT DEFAULT(d) FROM t returns NULL. branch-4.1 and branch-4.2 (apache#66538)
resolve all of these against the pinned write target, by name against the
target in INSERT ... SELECT, and reject DEFAULT(column) on an Iceberg column
in a statement that does not write the table.

This change adds the connector capability SUPPORTS_WRITE_ONLY_DEFAULTS,
declared by the Iceberg connector and inherited by iceberg-on-HMS tables.
For a table that declares it, InsertUtils resolves DEFAULT(column) in INSERT
VALUES and in the select lists of INSERT ... SELECT by name against the
pinned writer schema, and RewriteDefaultExpression takes the default of a
DEFAULT(column) bound to the table from the pinned writer column with the
same field id, which covers the UPDATE and MERGE conditions, and rejects it
when the statement pinned no writer schema for the table. Hive, Paimon and
OLAP tables keep the current behavior. The new enum constant changes the
connector plugin API surface, so the plugin API version goes from 26.0 to
27.0.

### Release note

Fix: On Iceberg tables, DEFAULT(column) in INSERT VALUES, INSERT ... SELECT
and the WHERE / ON conditions of UPDATE and MERGE now resolves to the
column's write-default. Before, it failed, wrote NULL, or matched no row.
SELECT DEFAULT(column) on an Iceberg table now fails instead of returning
NULL.

### Check List (For Author)

- Test: Regression test / Unit Test
    - test_iceberg_initial_defaults: writes rows 20-22 with
      DEFAULT(default_int) in VALUES and in INSERT ... SELECT with and
      without FROM, then updates them with UPDATE ... WHERE default_int =
      DEFAULT(default_int) and MERGE ... ON ... = DEFAULT(...), for Parquet
      and ORC; SELECT DEFAULT(default_int) must fail. Passes locally; with
      master's FE it fails at the first INSERT with "Unknown column
      'default_int'"
    - InsertUtilsTest: VALUES and INSERT ... SELECT (from a staging relation
      and without FROM) resolve against the pinned writer schema; unknown
      column fails; a table without the capability is unchanged
    - RewriteDefaultExpressionTest: pinned writer column wins, no pinned
      schema fails, a table without the capability keeps the read default
    - IcebergConnectorTest, HiveConnectorMetadataSiblingDelegationTest,
      ConnectorPluginSurfaceTest
- Behavior changed: Yes. See the release note.
- Does this need documentation: No

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
morningman added a commit that referenced this pull request Nov 2, 2026
…hema (#69488)

### What problem does this PR solve?

Issue Number: None

Related PR: #66538, #57531, #65851

Problem Summary:

**In short.** On an Iceberg table, `DEFAULT(column)` anywhere except
`UPDATE ... SET` and `MERGE ... VALUES` reads the default from the read
schema, which carries no Iceberg write-default. This PR resolves it
against the write schema that the statement pinned for its target, as
branch-4.1 and branch-4.2 do.

**Background.** An Iceberg `write-default` applies to rows written after
it is set. So it lives in the writer schema that a write statement pins
(`IcebergWriteSchemaContext`), not in the columns of the read schema.
#65851 resolves `DEFAULT(column)` against that pinned schema in `UPDATE
... SET`, `MERGE ... UPDATE SET` and `MERGE ... INSERT VALUES`
(`ConnectorWriteSchemaUtils`). Every other `DEFAULT(column)` either
fails at binding or reaches `RewriteDefaultExpression` (#57531), which
reads the default from the bound column. branch-4.1 #66538, also in
branch-4.2, resolves these against the pinned write target too. It
rejects `DEFAULT(column)` on an Iceberg column in a statement that does
not write the table.

**The problem, and what it cost.** Reproduced on a local cluster with a
`file://` format-version 3 table: `CREATE TABLE t (id INT, c INT)`,
`ALTER TABLE t ADD COLUMN d INT DEFAULT 42`, one row `(0, 0)`, so `d =
42`.

| Statement | Master today | After this PR (same as branch-4.2) |
|---|---|---|
| `INSERT INTO t (id, c, d) VALUES (1, 1, DEFAULT(d))` | `Unknown column
'd' in 'table list'` | writes 42 |
| `INSERT INTO t (id, c, d) SELECT 2, 2, DEFAULT(d) FROM t WHERE id = 0`
| writes NULL | writes 42 |
| `INSERT INTO t (id, c, d) SELECT 3, 3, DEFAULT(d)` | `Unknown column
'd'` | writes 42 |
| `UPDATE t SET c = 100 WHERE d = DEFAULT(d)` | updates nothing: `d =
NULL` makes the plan an empty set | updates every row, `d = 42` is
pushed down to the scan |
| `MERGE INTO t USING (SELECT 0 AS id) s ON t.id = s.id AND t.d =
DEFAULT(t.d) WHEN MATCHED THEN UPDATE SET c = 300` | matches nothing |
updates row 0 |
| `SELECT DEFAULT(d) FROM t` | NULL | `DEFAULT(d) must reference a
column of the table that this statement writes` |

The `UPDATE` and `MERGE` cases fail silently: the statement succeeds and
changes nothing.

**How this PR fixes it.**

- fe-core cannot tell from a column whether its connector keeps defaults
in the read schema. Hive does: an HMS default shows up in both schemas.
Iceberg does not. The new connector capability
`SUPPORTS_WRITE_ONLY_DEFAULTS` says which kind a table is. The Iceberg
connector declares it. The Hive connector passes it on to iceberg-on-HMS
tables, like the other per-table capabilities, but only through the
two-argument `getTableSchema`. Since #69299, a statement that pins the
table at its latest snapshot reads the schema from the snapshot
overload, which reflects none of them, so on an iceberg-on-HMS table
such a statement keeps master's results (for example, `UPDATE t SET c =
100 WHERE d = DEFAULT(d)` still changes no row). Reflecting the
sibling's capabilities there changes planning for all of them and is
left to a follow-up PR.
- For a target that declares it, `InsertUtils.normalizePlan` resolves
`DEFAULT(column)` in `INSERT ... VALUES` and in the select lists of
`INSERT ... SELECT`, including the replacements of `SELECT *
REPLACE(...)`, against the pinned write schema. It does this by name,
before the source relations are bound, so the name always refers to a
target column, as in branch-4.2. In MySQL, `DEFAULT(column)` in a
`SELECT` refers to a source column instead. A derived table in `FROM` is
a query of its own and is left alone: `DEFAULT(column)` in it still
names a column of its own `FROM` clause.
- `RewriteDefaultExpression` takes the default for a `DEFAULT(column)`
bound to such a table from the pinned write column with the same field
id. That covers the `WHERE` of `UPDATE` and the `ON` of `MERGE`. A MERGE
resolves it whatever its WHEN branches, also when they all delete, as
branch-4.2 does. If the statement pinned no write schema for that table,
it fails. A `DELETE` pins the writer schema too, for its delete sink,
but it writes no row values, so `RowLevelDmlCommand` marks its target
and `DEFAULT(column)` on it fails with `DEFAULT(d) cannot be used in a
DELETE: the defaults of this table apply only to the rows that a
statement writes`, as in branch-4.2, whose DELETE installs no write
schema context. Master reads NULL there, so `DELETE FROM t WHERE d =
DEFAULT(d)` deletes nothing; resolved to the write-default it would
delete every row with `d = 42`.
- A resolved default is cast to the type of its writer column before the
expression around it is bound, as `RewriteDefaultExpression` already
does for a bound `DEFAULT(column)`. The parsed literal `-1` is a
TINYINT, so without the cast `bit_count(DEFAULT(c))` on an INT column
would count 8 bits instead of 32. The external `UPDATE`, `MERGE` and
changelog paths use the same helper. `RewriteDefaultExpression` rejects
a bound `DEFAULT(column)` whose writer column has another type than the
read column the statement was bound with, which happens only when the
table changed outside Doris after Doris cached it, and asks to refresh
the table: the expressions around it were bound against the read type.
- Hive, Paimon and internal tables do not declare the capability, so
they behave as before.
- The new enum constant changes the connector plugin API surface, so
`connector.plugin.api.version` goes from 26.0 to 27.0, as for the other
recent changes to that surface (#69299, #69588, #69298, #69683 and
#69835 took 22.0 to 26.0).

**Where `DEFAULT(column)` resolves on an Iceberg target (direct Iceberg
catalogs).**

| Statement | Position of `DEFAULT(column)` | Result | branch-4.2 |
|---|---|---|---|
| `INSERT` / `INSERT OVERWRITE` | `VALUES` rows | the target column's
write-default, typed as the column | same |
| | the INSERT's own select lists (with or without `FROM`, set operation
branches, `SELECT * REPLACE`) | the target column's write-default, by
name, before binding | same, except `* REPLACE`, which branch-4.2 binds
to the source |
| | `GROUP BY` / `ROLLUP` outputs, a derived table in `FROM`, `WITH`
definitions, subqueries, `WHERE` / `ON` | bound like any
`DEFAULT(column)` (last row) | same, except derived tables, which
branch-4.2 resolves against the target |
| | a `VALUES` branch of a set operation | fails with an unknown column,
as on master | same |
| | a qualifier other than the target (`DEFAULT(src.c)`,
`DEFAULT(ctl.db.t.c)`) in the positions above | fails: `column in
assignment list is invalid` | same |
| `UPDATE` | `SET` / `WHERE` | write-default | same |
| `MERGE` | `ON` and `WHEN ... AND` conditions, whatever the WHEN
branches, also when they all delete | write-default | same |
| | `UPDATE SET` / `INSERT VALUES` | write-default | same |
| `DELETE` (also `DELETE ... USING`) | anywhere | fails: `cannot be used
in a DELETE` | fails: its DELETE has no write schema context |
| any | a bound `DEFAULT(x.c)` | `x` is the written table: its
write-default by field id; an OLAP or Hive table: that column's default;
another Iceberg table: fails with `must reference a column of the table
that this statement writes` | same |

A bound `DEFAULT(column)` whose writer column has another type than the
read column fails and asks to refresh the table. Where `DEFAULT(column)`
is bound rather than resolved by name (the third row), it keeps its
meaning from before this PR and names a column of its own `FROM` clause,
as on master and branch-4.2: for example in `INSERT INTO t SELECT id,
DEFAULT(d) FROM src GROUP BY id, d` with an OLAP or Hive `src`, it takes
the default of `src.d`.

**Results.**

- Local cluster, with the table above: the table shows what each
statement did before and after. `EXPLAIN UPDATE t SET c = 5 WHERE d =
DEFAULT(d)` goes from an empty set to a scan with `d = 42` pushed down
to Iceberg.
- Unit tests: 6 new tests in `InsertUtilsTest`, 6 in the new
`RewriteDefaultExpressionTest` and 1 in `RowLevelDmlCommandTest`. They
cover VALUES, and INSERT ... SELECT both from a staging relation whose
own column is also `name` and without FROM: each resolves to the pinned
default, and an unknown column fails. `DEFAULT(column)` inside a derived
table, qualified or not and also under a UNION branch, is left to
binding. A `SELECT * REPLACE(DEFAULT(name) AS name)` replacement
resolves like the rest of the select list and keeps the star's qualifier
and `EXCEPT` list. The pinned write column wins over the read column. A
statement that writes nothing fails. A table without the capability is
unchanged. With master's `InsertUtils` and `RewriteDefaultExpression`,
the 5 tests of the fix fail and the 2 tests of the unchanged path pass.
The `SELECT * REPLACE` test fails with this PR's previous `InsertUtils`,
where the replacement stays `DEFAULT(name)`. `IcebergConnectorTest`,
`HiveConnectorMetadataSiblingDelegationTest` and
`ConnectorPluginSurfaceTest` pass. So do the fe-core tests around the
write schema: `ConnectorWriteSchemaUtilsTest`,
`ExternalRowLevelUpdatePlanBuilderTest`,
`ExternalRowLevelMergePlanBuilderTest`,
`ChangelogRowLevelDmlTransformTest` and `PluginDrivenExternalTableTest`.
Two of the tests pin the typing: `bit_count(DEFAULT(bits))` gets the
default as an INT in VALUES and in SELECT, and fails without the cast; a
BIGINT writer column over an INT read column is rejected, directly and
under an IF bound to INT. `DEFAULT(column)` on the target of a DELETE
fails even with a pinned write-default, and only a DELETE marks its
target (an UPDATE does not); skipping the check, or marking UPDATE
instead of DELETE, makes exactly these two tests fail.
`PositionDeleteRowLevelDmlTransformTest`,
`PhysicalPlanTranslatorIcebergRowLevelDmlTest`, `DeleteFromCommandTest`
and `UpdateCommandTest` pass.
- Regression: `test_iceberg_initial_defaults` now writes rows 20 to 22
with `DEFAULT(default_int)` in VALUES and in INSERT ... SELECT with and
without FROM. It then updates them with `WHERE default_int =
DEFAULT(default_int)` and `MERGE ... ON ... = DEFAULT(...)`, for Parquet
and ORC, and checks the rows in the `.out`. A plain `SELECT
DEFAULT(default_int)` must fail, and so must a `DELETE ... WHERE
default_int = DEFAULT(default_int)` on rows 20 to 22, which hold the
write-default. The suite passed locally before the DELETE block was
added, and the whole suite, DELETE block included, has since passed in
two External regression runs on CI. With master's FE it fails at the
first of these statements: `Unknown column 'default_int' in 'table
list'`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.x reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants