Repository navigation
Commit abca306
authored
Prevent Duplicate Sink Operator Generation for Shared Storage URIs (#3264)
This PR addresses an issue in the workflow scheduler where duplicate
sink operators are generated when multiple blocking downstream operators
cause the split operator to produce multiple sinks with the same storage
URI. With these changes, an additional check is added during the
physical plan construction. If a sink operator with the given URI
already exists, the scheduler will skip generating another one, thereby
preventing redundancy.
The PR will fix the issue #3263.
<img width="1528" alt="Screenshot 2025-02-13 at 4 22 20 PM"
src="https://github.com/user-attachments/assets/c8061e3a-50a7-4acf-a518-04cde6af9090"
/>1 parent af8d2d0 commit abca306
1 file changed
Lines changed: 38 additions & 26 deletions
File tree
- core/amber/src/main/scala/edu/uci/ics/amber/engine/architecture/scheduling
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
3 | 4 | | |
4 | 5 | | |
5 | 6 | | |
| |||
151 | 152 | | |
152 | 153 | | |
153 | 154 | | |
154 | | - | |
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| |||
166 | 166 | | |
167 | 167 | | |
168 | 168 | | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
173 | | - | |
174 | | - | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
175 | 205 | | |
176 | | - | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
182 | | - | |
183 | | - | |
184 | | - | |
185 | | - | |
186 | | - | |
187 | | - | |
188 | | - | |
189 | | - | |
190 | | - | |
191 | | - | |
192 | | - | |
193 | | - | |
194 | | - | |
| 206 | + | |
195 | 207 | | |
196 | 208 | | |
197 | 209 | | |
| |||
0 commit comments