Skip to content

Commit 63b2f2a

Browse files
committed
[test](regression) Load complex ORC fixtures through S3 TVF
### What problem does this PR solve? Issue Number: N/A Related PR: #69593 Problem Summary: The ARRAY and MAP regression fixtures were migrated from LOCAL TVF to ORC Stream Load, but the load scanner rejects complex destination columns. Upload the original fixtures to unique temporary S3 objects and use INSERT SELECT from S3 TVF, cleaning up each object after the insert. Preserve table definitions, synchronization and all existing count/value checks. ### Release note None ### Check List (For Author) - Test: Groovy 4.0.19 syntax parsing, ORC fixture schema/row-count/content inspection and git diff --check passed. Runtime regression was not executed locally; matching FE/BE artifacts are unavailable in these task worktrees. - Behavior changed: No (test data-loading adaptation only) - Does this need documentation: No
1 parent 6280fc5 commit 63b2f2a

2 files changed

Lines changed: 30 additions & 8 deletions

File tree

‎regression-test/suites/query_p0/test_array_with_single_row_max_than_uint16.groovy‎

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,10 +36,21 @@ suite("test_array_with_single_row_max_than_uint16", "query") {
3636
)
3737
"""
3838
// prepare data
39-
streamLoad {
40-
table "${testTable}"
41-
set 'format', 'orc'
42-
file "${context.config.dataPath}/query_p0/arr_max.orc"
39+
String bucket = getS3BucketName()
40+
String key = "regression/local_tvf_migration/${UUID.randomUUID()}/arr_max.orc"
41+
def s3Client = getS3Client()
42+
s3Client.putObject(bucket, key, new File(context.config.dataPath, "query_p0/arr_max.orc"))
43+
try {
44+
sql """INSERT INTO ${testTable} SELECT k1, k2, c_decimal FROM S3(
45+
"uri" = "s3://${bucket}/${key}",
46+
"format" = "orc",
47+
"s3.endpoint" = "${getS3Endpoint()}",
48+
"s3.region" = "${getS3Region()}",
49+
"s3.access_key" = "${getS3AK()}",
50+
"s3.secret_key" = "${getS3SK()}"
51+
)"""
52+
} finally {
53+
s3Client.deleteObject(bucket, key)
4354
}
4455
sql "sync"
4556
qt_sql "SELECT COUNT(*) FROM ${testTable}"

‎regression-test/suites/types/complex_types/map_uniq_with_local_tvf.groovy‎

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,21 @@ suite("map_uniq_with_local_tvf", "p0") {
3434
"disable_auto_compaction" = "false"
3535
); """
3636

37-
streamLoad {
38-
table "${table_name}"
39-
set 'format', 'orc'
40-
file "${context.config.dataPath}/types/complex_types/mm.orc"
37+
String bucket = getS3BucketName()
38+
String key = "regression/local_tvf_migration/${UUID.randomUUID()}/mm.orc"
39+
def s3Client = getS3Client()
40+
s3Client.putObject(bucket, key, new File(context.config.dataPath, "types/complex_types/mm.orc"))
41+
try {
42+
sql """INSERT INTO ${table_name} SELECT id, m FROM S3(
43+
"uri" = "s3://${bucket}/${key}",
44+
"format" = "orc",
45+
"s3.endpoint" = "${getS3Endpoint()}",
46+
"s3.region" = "${getS3Region()}",
47+
"s3.access_key" = "${getS3AK()}",
48+
"s3.secret_key" = "${getS3SK()}"
49+
)"""
50+
} finally {
51+
s3Client.deleteObject(bucket, key)
4152
}
4253
sql "sync"
4354
qt_sql "SELECT COUNT(*) FROM ${table_name}"

0 commit comments

Comments
 (0)