Repository navigation
Commit 6807723
[fix](fe) Make routine load ALTER atomic and survive replay failures
### What problem does this PR solve?
Issue Number: None
Related PR: #66634
Problem Summary: Two failure paths of persisting the ALTER ROUTINE LOAD
definition could stop FE or diverge master and followers.
1. Replay failure stopped journal replay. replayLoadDefinition wrapped
UserException in IllegalStateException, and runtime exceptions such as the
ParseException of the persisted ALTER, NoSuchElementException from
getDb(dbId).get(), or the nereids AnalysisException of an expression passed
through unchanged. Kafka and Kinesis replayModifyProperties only catch
UserException, so the exception escaped journal replay and the master restart
or follower replay failed. A realistic trigger is an upgrade whose analysis
rules (function signatures, type inference, configs) reject an ALTER written
by the previous version. Replay now follows the gsonPostProcess convention:
catch the exception, cancel the job with a warning, and remove its txn
callback. Nothing is applied before analysis and statement building succeed,
so the job keeps its previous definition. The database is looked up with
getDbOrMetaException instead of Optional.get().
2. Master ALTER was not atomic. modifyProperties modified job properties, the
load description and session variables before mergeLoadDescToOriginStatement,
which calls getTableName(), the parser and Preconditions.checkState. If that
step threw, the in-memory job had changed but no edit log was written, so the
master ran the new definition while followers and a restarted master kept the
old one. The step is split into mergeLoadDesc (overlay the ALTER clauses on
the current definition without touching the job), buildLoadDefinitionStatement
(build and verify the CREATE statement) and applyLoadDefinition (install it).
modifyProperties builds and verifies the new statement before modifying the
job, so a failure leaves the job and the journal unchanged. Replay uses the
same helpers.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- New RoutineLoadJobPersistenceTest cases for Kafka and Kinesis: replay of an unparseable ALTER, an unanalyzable expression, a missing database and a missing table cancels the job without throwing; a master ALTER that fails to build the persisted statement leaves job properties, load definition, session variables, origStmt and the edit log unchanged. All 10 fail against the previous implementation.
- ./run-fe-ut.sh --run RoutineLoadJobPersistenceTest, KafkaRoutineLoadJobTest, KinesisRoutineLoadJobTest, AlterRoutineLoadOperationLogTest, RoutineLoadJobTest, RoutineLoadManagerTest: 89 tests, 0 failures, 0 errors
- BUILD_TYPE=ASAN ./build.sh --fe: passed, including checkstyle
- The docker regression suite test_routine_load_alter_checkpoint_restart_fe was not run
- Behavior changed: Yes. An ALTER ROUTINE LOAD journal entry whose load clauses can not be replayed cancels the job on that FE instead of stopping FE startup or replay, the same as an unrestorable job in the image.
- Does this need documentation: No
Co-Authored-By: Claude <[email protected]>1 parent 79006c1 commit 6807723
4 files changed
Lines changed: 238 additions & 51 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris/load/routineload
- kafka
- kinesis
- test/java/org/apache/doris/load/routineload
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2048 | 2048 | | |
2049 | 2049 | | |
2050 | 2050 | | |
2051 | | - | |
2052 | | - | |
2053 | | - | |
2054 | | - | |
2055 | | - | |
2056 | | - | |
2057 | | - | |
2058 | | - | |
2059 | | - | |
2060 | | - | |
2061 | | - | |
2062 | | - | |
2063 | | - | |
| 2051 | + | |
2064 | 2052 | | |
2065 | 2053 | | |
2066 | 2054 | | |
2067 | 2055 | | |
2068 | 2056 | | |
2069 | 2057 | | |
2070 | 2058 | | |
| 2059 | + | |
| 2060 | + | |
| 2061 | + | |
| 2062 | + | |
| 2063 | + | |
| 2064 | + | |
| 2065 | + | |
| 2066 | + | |
| 2067 | + | |
| 2068 | + | |
| 2069 | + | |
| 2070 | + | |
| 2071 | + | |
| 2072 | + | |
| 2073 | + | |
2071 | 2074 | | |
2072 | 2075 | | |
2073 | 2076 | | |
2074 | 2077 | | |
2075 | 2078 | | |
2076 | | - | |
2077 | | - | |
2078 | | - | |
2079 | | - | |
2080 | 2079 | | |
2081 | | - | |
2082 | | - | |
2083 | | - | |
2084 | | - | |
2085 | | - | |
2086 | | - | |
2087 | | - | |
| 2080 | + | |
| 2081 | + | |
| 2082 | + | |
| 2083 | + | |
| 2084 | + | |
| 2085 | + | |
| 2086 | + | |
| 2087 | + | |
| 2088 | + | |
| 2089 | + | |
| 2090 | + | |
| 2091 | + | |
| 2092 | + | |
| 2093 | + | |
| 2094 | + | |
| 2095 | + | |
| 2096 | + | |
2088 | 2097 | | |
2089 | | - | |
2090 | | - | |
2091 | | - | |
2092 | | - | |
| 2098 | + | |
| 2099 | + | |
| 2100 | + | |
| 2101 | + | |
| 2102 | + | |
| 2103 | + | |
| 2104 | + | |
2093 | 2105 | | |
2094 | 2106 | | |
2095 | 2107 | | |
| |||
2119 | 2131 | | |
2120 | 2132 | | |
2121 | 2133 | | |
2122 | | - | |
2123 | | - | |
2124 | | - | |
2125 | | - | |
2126 | | - | |
| 2134 | + | |
| 2135 | + | |
| 2136 | + | |
| 2137 | + | |
| 2138 | + | |
| 2139 | + | |
| 2140 | + | |
| 2141 | + | |
| 2142 | + | |
| 2143 | + | |
| 2144 | + | |
| 2145 | + | |
| 2146 | + | |
| 2147 | + | |
| 2148 | + | |
| 2149 | + | |
| 2150 | + | |
| 2151 | + | |
| 2152 | + | |
| 2153 | + | |
| 2154 | + | |
| 2155 | + | |
| 2156 | + | |
| 2157 | + | |
| 2158 | + | |
| 2159 | + | |
| 2160 | + | |
| 2161 | + | |
2127 | 2162 | | |
2128 | 2163 | | |
2129 | 2164 | | |
2130 | 2165 | | |
2131 | 2166 | | |
2132 | | - | |
| 2167 | + | |
2133 | 2168 | | |
2134 | 2169 | | |
2135 | 2170 | | |
2136 | 2171 | | |
2137 | 2172 | | |
2138 | 2173 | | |
2139 | | - | |
2140 | | - | |
2141 | | - | |
2142 | | - | |
| 2174 | + | |
| 2175 | + | |
| 2176 | + | |
| 2177 | + | |
2143 | 2178 | | |
2144 | 2179 | | |
2145 | | - | |
| 2180 | + | |
| 2181 | + | |
| 2182 | + | |
| 2183 | + | |
| 2184 | + | |
| 2185 | + | |
| 2186 | + | |
2146 | 2187 | | |
2147 | 2188 | | |
2148 | 2189 | | |
2149 | 2190 | | |
| 2191 | + | |
2150 | 2192 | | |
2151 | 2193 | | |
2152 | 2194 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| 45 | + | |
45 | 46 | | |
46 | 47 | | |
47 | 48 | | |
| |||
57 | 58 | | |
58 | 59 | | |
59 | 60 | | |
| 61 | + | |
60 | 62 | | |
61 | 63 | | |
62 | 64 | | |
| |||
790 | 792 | | |
791 | 793 | | |
792 | 794 | | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
793 | 804 | | |
794 | 805 | | |
795 | | - | |
796 | | - | |
797 | | - | |
| 806 | + | |
| 807 | + | |
798 | 808 | | |
799 | 809 | | |
800 | 810 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| 38 | + | |
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
| |||
50 | 51 | | |
51 | 52 | | |
52 | 53 | | |
| 54 | + | |
53 | 55 | | |
54 | 56 | | |
55 | 57 | | |
| |||
686 | 688 | | |
687 | 689 | | |
688 | 690 | | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
689 | 700 | | |
690 | 701 | | |
691 | | - | |
692 | | - | |
693 | | - | |
| 702 | + | |
| 703 | + | |
694 | 704 | | |
695 | 705 | | |
696 | 706 | | |
| |||
0 commit comments