Repository navigation
Commit bd2b144
authored
Prevent Duplicate Sink Operator Generation for Shared Storage URIs (apache#3264)
This PR addresses an issue in the workflow scheduler where duplicate
sink operators are generated when multiple blocking downstream operators
cause the split operator to produce multiple sinks with the same storage
URI. With these changes, an additional check is added during the
physical plan construction. If a sink operator with the given URI
already exists, the scheduler will skip generating another one, thereby
preventing redundancy.
The PR will fix the issue apache#3263.
<img width="1528" alt="Screenshot 2025-02-13 at 4 22 20 PM"
src="https://github.com/user-attachments/assets/c8061e3a-50a7-4acf-a518-04cde6af9090"
/>1 parent 96553a9 commit bd2b144
1 file changed
Lines changed: 38 additions & 26 deletions
File tree
- core/amber/src/main/scala/edu/uci/ics/amber/engine/architecture/scheduling
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
3 | 4 | | |
4 | 5 | | |
5 | 6 | | |
| |||
151 | 152 | | |
152 | 153 | | |
153 | 154 | | |
154 | | - | |
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| |||
166 | 166 | | |
167 | 167 | | |
168 | 168 | | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
173 | | - | |
174 | | - | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
175 | 205 | | |
176 | | - | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
182 | | - | |
183 | | - | |
184 | | - | |
185 | | - | |
186 | | - | |
187 | | - | |
188 | | - | |
189 | | - | |
190 | | - | |
191 | | - | |
192 | | - | |
193 | | - | |
194 | | - | |
| 206 | + | |
195 | 207 | | |
196 | 208 | | |
197 | 209 | | |
| |||
0 commit comments