You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit ccf1079
Browse filesBrowse the repository at this point in the historyBrowse files
feat(amber): bind $K loop-variable references in operators
A property of any operator inside a control block may name a loop variable
as `$K`. The frontend keeps sending the literal string, whatever the
property's type; the backend now handles it in two steps.
At plan-parse time a Jackson module (StateReferenceModule, registered on
JSONUtils.objectMapper and on the three Dropwizard mappers) wraps the
deserializer of every StateReferencing bean -- LogicalOp now is one. An
object that holds a `$K` (or a client-sent sidecar) is rewritten: each
`$K` leaf becomes a placeholder of the property's JSON type (0 / 0.0 /
false; a String keeps the literal) and the property's JSON pointer ->
variable name is recorded in the `stateReferences` sidecar, which the
property panel never shows. Every other object is re-fed untouched from a
token buffer.
At the worker, ExecFactory hands out a LateBoundExecutor whenever the
descString carries a sidecar or a literal `$K`. It stays unbound until the
first state message, then patches the placeholders with the loop
variables (coerced to the placeholder's JSON type), instantiates and opens
the real executor once, and delegates every later call to it. A source is
never wrapped, since no state message reaches it. The worker registers
each state message on the executor (`registerState` / `loopState`) before
`processState` runs, and a reconfiguration or a checkpoint restore hands
the replacement executor the state its predecessor consumed.
An operator whose executor runs generated code (the UDFs and the
Python-generating descriptors) cannot bind a reference yet: the compiler
reports it instead of running with the placeholders.
Closesapache#8635
Copy file name to clipboardExpand all lines: amber/src/main/scala/org/apache/texera/amber/engine/architecture/worker/managers/SerializationManager.scala
Copy file name to clipboardExpand all lines: amber/src/main/scala/org/apache/texera/amber/engine/architecture/worker/promisehandlers/PrepareCheckpointHandler.scala
Copy file name to clipboardExpand all lines: amber/src/main/scala/org/apache/texera/amber/engine/architecture/worker/promisehandlers/UpdateExecutorHandler.scala
Copy file name to clipboardExpand all lines: amber/src/test/scala/org/apache/texera/amber/engine/architecture/worker/DataProcessorRPCHandlerInitializerSpec.scala
0 commit comments