You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit cf75462
Browse filesBrowse the repository at this point in the historyBrowse files
refactor(amber): count a materialized input port as completed once its END is processed (apache#8938)
### What changes were proposed in this PR?
`InputManager.isPortCompleted` now treats a port read from
materialization the same as a port fed by a link: it counts as completed
once END_CHANNEL has been processed on it. `EndChannelHandler` marks
that by setting `WorkerPort.completed`.
Before, such a port reported its reader thread's `finished` flag. The
thread sets that flag only after enqueuing END. So a DP thread waiting
on the queue, typically the worker that got no tuples, could run
`EndChannelHandler` before the flag was set. It then saw the port as not
completed and skipped `finalizeOutput`. Nothing called it again, so the
worker never finished and its region hung until the test deadline. That
is the intermittent `TimeoutException: 1.minutes+30.seconds` in
`LoopIntegrationSpec`.
```
Before: reader enqueues END -> DP processes END, flag not set yet -> output never finalized -> region hangs
After: reader enqueues END -> DP processes END, port completed -> output finalized -> region completes
```
Two related effects of the change:
- The flag could also be set while END was still queued. A worker with a
second materialized input no longer finalizes before that port's END has
been processed.
- `startInputPortReaderThreads` uses the same check to skip ports that
already completed. The second `StartWorker` of a two-phase region
arrives only after the coordinator has the port's `PortCompleted`, which
the worker sends after processing END. So it skips the same ports as
before.
- Nothing reads the reader thread's `finished` flag any more, so it is
removed, and it can't stand in for port completion again. The
reader-thread tests that read it now check the END_CHANNEL the reader
enqueues instead.
### Any related issues, documentation, discussions?
Closesapache#8937. The two `LoopIntegrationSpec` cases added in apache#8639 hit the
race most often; the race itself dates from apache#3460.
### How was this PR tested?
Tests first: the first commit adds the tests and fails on `main`, and
the second commit makes them pass.
- `InputManagerSpec`: a materialized port is completed once END has been
processed, even before its reader thread has returned. It is not
completed while END is still queued, even after the thread has returned.
- `EndChannelHandlerSpec`: when END arrives on a materialized input port
before its reader thread has returned, the worker emits the output
`FinalizePort` and `FinalizeExecutor`. On `main` it emits only the input
port's `FinalizePort`, which is the hang.
```
sbt "WorkflowExecutionService/testOnly *InputManagerSpec *EndChannelHandlerSpec *StartHandlerSpec *DataProcessorSpec *EndHandlerSpec *StartChannelHandlerSpec *AssignPortHandlerSpec *DPThreadSpec *WorkerSpec"
sbt "WorkflowExecutionService/scalafmtCheckAll" "WorkflowExecutionService/scalafixAll --check"
```
`InputPortMaterializationReaderThreadSpec` reads real Iceberg tables.
Locally only its two cases that don't need storage ran; CI runs the
rest.
`LoopIntegrationSpec`, run locally on Windows with the sbt JVM and its
Python workers pinned to 3 CPUs (affinity mask `0x7`), one case per JVM:
| Case | `main` | This PR |
|---|---|---|
| bind a `$i` reference after two branches of the loop body meet again |
3 of 5 runs hung on the first attempt; 1 also hung on the retry and
failed as on CI | 10 of 10 passed on the first attempt, 42–44 s each |
| bind the inner loop's variable where it shadows the outer loop's |
(fails on CI, see apache#8937) | 10 of 10 passed on the first attempt, 64–70 s
each |
Unpinned, the whole spec passes with this PR: 8 of 8 cases, 29–59 s
each, no retries.
```
AMBER_TEST_FILTER= STORAGE_ICEBERG_CATALOG_TYPE=postgres sbt "WorkflowExecutionService/testOnly *LoopIntegrationSpec -- -z \"after two branches of the loop body meet again\""
AMBER_TEST_FILTER=integration-only STORAGE_ICEBERG_CATALOG_TYPE=postgres sbt "WorkflowExecutionService/testOnly *LoopIntegrationSpec"
```
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Copy file name to clipboardExpand all lines: amber/src/main/scala/org/apache/texera/amber/engine/architecture/worker/managers/InputPortMaterializationReaderThread.scala
Copy file name to clipboardExpand all lines: amber/src/test/scala/org/apache/texera/amber/engine/architecture/worker/managers/InputPortMaterializationReaderThreadSpec.scala
+18-27Lines changed: 18 additions & 27 deletions
Original file line number
Diff line number
Diff line change
@@ -157,20 +157,15 @@ class InputPortMaterializationReaderThreadSpec extends AnyFlatSpec {
157
157
hashAttributeNames =Seq("id")
158
158
)
159
159
160
-
/**
161
-
* Runs the reader inline and returns everything it enqueued, in order, together with the reader
162
-
* itself so its completion flag can be read.
163
-
*/
160
+
/** Runs the reader inline and returns everything it enqueued, in order. */
Copy file name to clipboardExpand all lines: amber/src/test/scala/org/apache/texera/amber/engine/architecture/worker/promisehandlers/EndChannelHandlerSpec.scala
0 commit comments