Skip to content

Commit 77459c4

Browse files
morningmanshuke987claude
committed
[test](regression) Stabilize test_auto_new_recycle and test_sql_digest_generation
### Background An auto-partition table with `partition.retention_count = N` keeps only its N latest historical partitions: the FE's `DynamicPartitionScheduler` drops older ones on every run, once per `dynamic_partition_check_interval_seconds` (600 by default). `partition_p0/auto_partition/test_auto_new_recycle` lowers that interval to 1 s and checks the partition count after each load or ALTER. `audit/test_sql_digest_generation` checks that the audit log gets an `sql_digest` only for queries slower than the FE config `sql_digest_generation_threshold_ms`, once with the threshold at 0 and once at 100000. Both suites are in group `nonConcurrent`. ### What problem does this PR solve? Issue Number: None Related PR: #65375 and #65646 (the branch-4.1 recycle-suite fixes by @shuke987, ported here), #65343 (the branch-4.1 digest-suite fix by @shuke987, ported here), #65435 (master's earlier move of the recycle suite's day-boundary guard) PR Type: Test Problem Summary: Both suites rely on timing the product does not promise. The recycle suite checks the partition count after a fixed `sleep(8000)` (5 s after an ALTER), while the drop happens asynchronously at an unbounded time after the config change; branch-4.1's NonConcurrent pipeline failed at the first such check repeatedly (#65375). The digest suite's threshold-0 check needs the query's elapsed time to be above 0 ms. This PR gives both suites their branch-4.1/4.2 form: the recycle suite polls the partition count every second, for at most 1200 s, and widens its day-boundary guard to cover the poll; the digest query adds `sleep(1)`. Both files become identical to branch-4.2. ### Details - **Failure:** On master's public NonConcurrent pipeline neither suite has failed on its own since mid-July: of about 8,000 runs each, the recycle suite failed 3 times and the digest suite 87 times, all in builds whose cluster had died (most suites failing). The failures that motivated the fixes are on branch-4.1's pipeline (#65375); #65343 made the digest query deterministic there. - **Cause:** `ADMIN SET FRONTEND CONFIG` sets the scheduler's new interval at once, but the scheduler sleeps in 5-second slices and notices the change only at the next slice. It then runs `executeDynamicPartition` over every registered table before the drop. So the drop lands 0 to 5 s plus one full pass after the config change, and a pass longer than about 3 s fails the 8-second check. In the digest suite, `AuditLogHelper` writes a digest only when `elapseMs > sql_digest_generation_threshold_ms`, and nothing bounds the elapsed time of `select *` on a two-row table from below (9 to 15 ms locally). With `sleep(1)` on each row it takes about 2 s, still far below the second check's 100000 ms. - **Stability evidence:** The recycle checks now wait for the drop and fail only when the count misses the target for 1200 s. The poll runs on the suite thread, because the framework sets `Awaitility.pollInSameThread()` (#68338), so it does not leak connections. Shortening one of master's waits to 1 s reproduces the branch-4.1 failure shape, and the ported suite passes; mutants show that both suites still catch wrong results (see Testing). ### Testing All runs used `./run-regression-test.sh --run -d partition_p0/auto_partition -s test_auto_new_recycle` or `--run -d audit -s test_sql_digest_generation` against a local single-FE, single-BE cluster built from master. | Suite | Version | Result | |---|---|---| | `test_auto_new_recycle` | master | passes | | `test_auto_new_recycle` | this PR | passes; the polls reached the target after 1, 2, 1, 3 and 3 s | | `test_auto_new_recycle` | master, with the wait before the 10-partition check cut to 1 s | fails as #65375 describes: `expected: <101> but was: <10>` (master's `assertEquals` has its arguments swapped, so 101 partitions were still there) | | `test_auto_new_recycle` | this PR, expecting 11 partitions, poll capped at 20 s | fails as it should: `ConditionTimeoutException` | | `test_sql_digest_generation` | master | passes | | `test_sql_digest_generation` | this PR | passes; the query took 2,024 and 2,025 ms | | `test_sql_digest_generation` | this PR, second threshold 1000 ms | fails as it should: `Count of sql_digest changed` | The PR NonConcurrent pipeline runs both suites; cloud_p0 skips `nonConcurrent` suites. ### Notes for Reviewers - The day-boundary guard before the final `now()`-based section goes from 100 s to 1300 s, as in #65646, so that it covers the 1200 s poll plus the insert and checks. It sleeps only when that section would otherwise start within 1300 s of midnight. - Four more suites fixed in the same branch-4.1 batch need nothing on master: #65340 (master #68449 enables the publish debug point on all FEs, superseding the open port #65556), #65929 (master #68329 fixed the actual cause, a connection leak), #65885 (master #69723 removed the V1 scanner whose message it also accepted), and #67363 (ported as #67402). ### Release note None ### Check List (For Author) - Behavior changed: No - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: Claude Code (Claude Opus 5.5) - Generated: the port (both suites taken from branch-4.2), the local verification runs, and this description - Human review: not stated - Prompts: The author asked to continue moving branch-4.2's changes back to master, where six branch-4.1 test fixes were recorded as one task. After the check showed that master already covers four of them and has not failed on the other two, the author chose to port those two exactly as branch-4.2 has them and to verify them on a local cluster. Co-authored-by: shuke <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 52362a4 commit 77459c4

2 files changed

Lines changed: 21 additions & 18 deletions

File tree

‎regression-test/suites/audit/test_sql_digest_generation.groovy‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ suite("test_sql_digest_generation", "nonConcurrent") {
4343
Long prevSqlDigestCount = Long.valueOf(prevSqlDigestCountResult[0][0])
4444
logger.info("prev sql_digest count: " + prevSqlDigestCount)
4545

46-
sql "select * from audit_log_behavior"
46+
sql "select *, sleep(1) from audit_log_behavior"
4747

4848
Thread.sleep(10000)
4949
sql """call flush_audit_log()"""
@@ -66,7 +66,7 @@ suite("test_sql_digest_generation", "nonConcurrent") {
6666
Long prevSqlDigestCount = Long.valueOf(prevSqlDigestCountResult[0][0])
6767
logger.info("prev sql_digest count: " + prevSqlDigestCount)
6868

69-
sql "select * from audit_log_behavior"
69+
sql "select *, sleep(1) from audit_log_behavior"
7070

7171
Thread.sleep(10000)
7272
sql """call flush_audit_log()"""

‎regression-test/suites/partition_p0/auto_partition/test_auto_new_recycle.groovy‎

Lines changed: 19 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,9 @@
1515
// specific language governing permissions and limitations
1616
// under the License.
1717

18+
import java.util.concurrent.TimeUnit
19+
import org.awaitility.Awaitility
20+
1821
suite("test_auto_new_recycle", "nonConcurrent") {
1922
sql "drop table if exists auto_recycle"
2023
test {
@@ -133,6 +136,16 @@ suite("test_auto_new_recycle", "nonConcurrent") {
133136
);
134137
"""
135138

139+
def waitPartitionCount = { int expectedCount ->
140+
Awaitility.await().atMost(1200, TimeUnit.SECONDS).pollInterval(1, TimeUnit.SECONDS).until {
141+
res = sql "show partitions from auto_recycle"
142+
logger.info("partition count: ${res.size()}, expected: ${expectedCount}")
143+
res.size() == expectedCount
144+
}
145+
res = sql "show partitions from auto_recycle"
146+
assertEquals(expectedCount, res.size())
147+
}
148+
136149
sql """
137150
insert into auto_recycle select date_add('2020-01-01 00:00:00', interval number day) from numbers("number" = "100");
138151
"""
@@ -144,9 +157,7 @@ suite("test_auto_new_recycle", "nonConcurrent") {
144157
assertTrue(res[0][1].contains('"partition.retention_count" = "3"'))
145158

146159
sql """ admin set frontend config ('dynamic_partition_check_interval_seconds' = '1') """
147-
sleep(8000)
148-
res = sql "show partitions from auto_recycle"
149-
assertEquals(res.size(), 3)
160+
waitPartitionCount.call(3)
150161

151162
// before everytime insert data, we should avoid recycle because if they conflict, insert will fail
152163
sql """ admin set frontend config ('dynamic_partition_check_interval_seconds' = '600') """
@@ -155,15 +166,11 @@ suite("test_auto_new_recycle", "nonConcurrent") {
155166
insert into auto_recycle select date_add('2020-01-01 00:00:00', interval number day) from numbers("number" = "100");
156167
"""
157168
sql """ admin set frontend config ('dynamic_partition_check_interval_seconds' = '1') """
158-
sleep(8000)
159-
res = sql "show partitions from auto_recycle"
160-
assertEquals(res.size(), 3)
169+
waitPartitionCount.call(3)
161170
qt_sql0 "select * from auto_recycle order by k0"
162171

163172
sql "alter table auto_recycle set ('partition.retention_count' = '1')"
164-
sleep(5000)
165-
res = sql "show partitions from auto_recycle"
166-
assertEquals(res.size(), 1)
173+
waitPartitionCount.call(1)
167174

168175
sql "alter table auto_recycle set ('partition.retention_count' = '10')"
169176
sql """ admin set frontend config ('dynamic_partition_check_interval_seconds' = '600') """
@@ -172,9 +179,7 @@ suite("test_auto_new_recycle", "nonConcurrent") {
172179
insert into auto_recycle select date_add('2022-01-01 00:00:00', interval number day) from numbers("number" = "100");
173180
"""
174181
sql """ admin set frontend config ('dynamic_partition_check_interval_seconds' = '1') """
175-
sleep(8000)
176-
res = sql "show partitions from auto_recycle"
177-
assertEquals(res.size(), 10)
182+
waitPartitionCount.call(10)
178183
qt_sql1 "select * from auto_recycle order by k0"
179184

180185
// verify not consider future partition when do recycle
@@ -191,14 +196,12 @@ suite("test_auto_new_recycle", "nonConcurrent") {
191196
"""
192197
sql """ admin set frontend config ('dynamic_partition_check_interval_seconds' = '600') """
193198
sleep(8000)
194-
waitUntilSafeExecutionTime("NOT_CROSS_DAY_BOUNDARY", 100)
199+
waitUntilSafeExecutionTime("NOT_CROSS_DAY_BOUNDARY", 1300)
195200
sql """
196201
insert into auto_recycle select date_add(now(), interval number-5 day) from numbers("number" = "8");
197202
"""
198203
sql """ admin set frontend config ('dynamic_partition_check_interval_seconds' = '1') """
199-
sleep(8000)
200-
res = sql "show partitions from auto_recycle"
201-
assertEquals(res.size(), 6) // [-5, -1] -> [-3, -1], [0, 2] -> [0, 2]
204+
waitPartitionCount.call(6) // [-5, -1] -> [-3, -1], [0, 2] -> [0, 2]
202205
res = sql "select * from auto_recycle order by k0"
203206
assertEquals(res.size(), 6)
204207

0 commit comments

Comments
 (0)