Skip to content

Commit 3fa608d

Browse files
authored
fix(amber): reject $i in settings the plan is built from (apache#8750)
### What changes were proposed in this PR? Root cause: inside a loop block, a loop-variable reference (apache#8639) is written into the operator's setting when the iteration's state arrives. The compiler, however, builds the operator's output schema and its partitioning earlier, when the workflow compiles, from the reference's placeholder. A reference in a property the plan is built from therefore never reaches the schema or the partitioning. ``` Before: Loop Start (a = "x") -> Projection (alias $a) -> Loop End output column named "$a", null in every row After: compile error: Projection cannot refer to loop variables (/attributes/0/alias -> $a): its output schema or partitioning is built from that property when the workflow is compiled, before the loop runs ``` | Piece | What it does | |---|---| | `@NoLoopVariable` | Marks a descriptor property that can't hold a loop variable because the plan is built from it: an output column's name or type, or what an input is partitioned on. `LogicalOp`'s documentation states the rule for operator authors. | | `StateReferencing.noLoopVariableProperties` / `noLoopVariableReferences` | Name the marked properties by their JSON names, and find the references under them at any depth. Both go through Jackson's bean introspection, so renamed JSON names such as `"result attribute"` resolve. | | `OperatorMetadataGenerator` | Puts `"noLoopVariable": true` on each marked property in the operator's JSON schema, from the same helper, so the panel (apache#8640) can use the same list. | | `WorkflowCompiler` | Inside a loop block, a reference under a marked property is a compile error, reported before the physical plan is built so it is the operator's first error. Outside every block nothing changes: a `$a` alias is still the literal column name. | | `ExecutionReconfigurationService` | Modifying an operator that refers to loop variables while the workflow runs is refused: `Limit refers to loop variables, so it can't be changed while the workflow runs.` Its rebuilt executor would never have the variables written into it. | Marked properties: | Operator | Properties | |---|---| | Projection | drop option, attributes | | Dictionary Matcher, Unnest String | result attribute | | Aggregate | aggregations, group-by keys | | File Scan From Input | attribute name, file-name column option (the attribute name is declared by the text-source trait, so File Scan and Text Input carry the mark too; as sources they are never inside a loop block) | | Type Casting | cast units | | Hash Join, Interval Join | the join attributes | | Sort Partitions | sort attribute, domain min and max | ### Any related issues, documentation, discussions? Closes apache#8749 Related to apache#8635 and apache#8639. ### How was this PR tested? New tests, each run red against the pre-change code first: | Spec | Covers | |---|---| | `StateReferenceModuleSpec` | `noLoopVariableProperties` names each marked property by its JSON name, a trait's included; `noLoopVariableReferences` picks the references under a marked property whether it is renamed, escaped, nested or declared by a trait, and nothing from a descriptor with no marked property or no reference. | | `OperatorMetadataGeneratorSpec` | The schema marks exactly the annotated properties (renamed ones under their JSON names) and no others, and for every operator the marked set equals `noLoopVariableProperties`. | | `WorkflowCompilerSpec` | Inside a loop block, a reference in each marked property is a compile error, and it comes before the operator's other errors. A reference in a property the executor reads, on an operator whose plan is built from other properties, still binds. Outside every block, a `$a` Projection alias stays the literal column name. | | `ExecutionReconfigurationServiceSpec` | An edit that makes an operator inside a loop block refer to a loop variable is refused, with the message above. An operator inside a block that refers to no loop variable, and a `$x` literal outside every block, can still be modified. | ``` sbt "WorkflowCore/testOnly org.apache.texera.amber.core.state.* org.apache.texera.amber.core.executor.*" # 75 passed sbt "WorkflowCompiler/test" # 83 passed sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.metadata.* ...projection.* ...aggregate.* \ ...sortPartitions.* ...dictionary.* ...unneststring.* ...typecasting.* ...hashJoin.* ...intervalJoin.*" # 467 passed sbt "WorkflowExecutionService/testOnly org.apache.texera.web.service.ExecutionReconfigurationServiceSpec" # 18 passed sbt "scalafmtCheckAll" "<WorkflowCore|WorkflowOperator|WorkflowCompiler|WorkflowExecutionService>/scalafixAll --check" # clean ``` ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
1 parent 05d48eb commit 3fa608d

20 files changed

Lines changed: 537 additions & 23 deletions

File tree

‎amber/src/main/scala/org/apache/texera/web/service/ExecutionReconfigurationService.scala‎

Lines changed: 39 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@
1919

2020
package org.apache.texera.web.service
2121

22+
import com.fasterxml.jackson.databind.node.ObjectNode
23+
import org.apache.texera.amber.core.state.StateReferencing.literalReferences
2224
import org.apache.texera.amber.core.storage.RepositoryMountManager
2325
import org.apache.texera.amber.core.virtualidentity.ActorVirtualIdentity
2426
import org.apache.texera.amber.engine.architecture.coordinator.{UpdateExecutorCompleted, Workflow}
@@ -27,6 +29,8 @@ import org.apache.texera.amber.engine.architecture.rpc.controlcommands.{
2729
WorkflowReconfigureRequest
2830
}
2931
import org.apache.texera.amber.engine.common.client.AmberClient
32+
import org.apache.texera.amber.operator.LogicalOp
33+
import org.apache.texera.amber.util.JSONUtils.objectMapper
3034
import org.apache.texera.web.SubscriptionManager
3135
import org.apache.texera.web.model.websocket.event.TexeraWebSocketEvent
3236
import org.apache.texera.web.model.websocket.request.ModifyLogicRequest
@@ -60,24 +64,47 @@ class ExecutionReconfigurationService(
6064
val newOp = modifyLogicRequest.operator
6165
val opId = newOp.operatorIdentifier
6266
val currentOp = workflow.logicalPlan.getOperator(opId)
63-
val reconfiguredPhysicalOp =
64-
currentOp.runtimeReconfiguration(
65-
workflow.context.workflowId,
66-
workflow.context.executionId,
67-
currentOp,
68-
newOp
67+
if (refersToLoopVariables(currentOp) || refersToLoopVariables(newOp)) {
68+
ModifyLogicResponse(
69+
opId.id,
70+
isValid = false,
71+
s"${currentOp.operatorInfo.userFriendlyName} refers to loop variables, so it can't be " +
72+
"changed while the workflow runs."
6973
)
70-
reconfiguredPhysicalOp match {
71-
case Failure(exception) => ModifyLogicResponse(opId.id, isValid = false, exception.getMessage)
72-
case Success(op) => {
73-
stateStore.reconfigurationStore.updateState(old =>
74-
old.copy(unscheduledReconfigurations = old.unscheduledReconfigurations :+ op)
74+
} else {
75+
val reconfiguredPhysicalOp =
76+
currentOp.runtimeReconfiguration(
77+
workflow.context.workflowId,
78+
workflow.context.executionId,
79+
currentOp,
80+
newOp
7581
)
76-
ModifyLogicResponse(opId.id, isValid = true, "")
82+
reconfiguredPhysicalOp match {
83+
case Failure(exception) =>
84+
ModifyLogicResponse(opId.id, isValid = false, exception.getMessage)
85+
case Success(op) => {
86+
stateStore.reconfigurationStore.updateState(old =>
87+
old.copy(unscheduledReconfigurations = old.unscheduledReconfigurations :+ op)
88+
)
89+
ModifyLogicResponse(opId.id, isValid = true, "")
90+
}
7791
}
7892
}
7993
}
8094

95+
/**
96+
* Whether `op`, in the deployed operator's place, refers to loop variables: its sidecar names
97+
* one (the compiler's final one on the deployed operator, the parse's typed ones on an edit),
98+
* or it sits inside a loop block and holds a whole-string `$name`. The rebuilt executor would
99+
* parse the placeholders after the loop state that writes them into its setting has arrived
100+
* (`OperatorExecutor.registerState`), so nothing would ever set them.
101+
*/
102+
private def refersToLoopVariables(op: LogicalOp): Boolean =
103+
op.stateReferences.nonEmpty || (
104+
workflow.logicalPlan.operatorsInsideLoopBlocks.contains(op.operatorIdentifier) &&
105+
literalReferences(objectMapper.valueToTree[ObjectNode](op)).nonEmpty
106+
)
107+
81108
// actually performs all reconfiguration requests the user made during pause
82109
// sends ModifyLogic messages to operators and workers,
83110
// see the Fries reconfiguration paper for the algorithm.

‎amber/src/test/scala/org/apache/texera/web/service/ExecutionReconfigurationServiceSpec.scala‎

Lines changed: 101 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ import org.apache.texera.amber.core.workflow.{
3737
OutputPort,
3838
PhysicalOp,
3939
PhysicalPlan,
40+
PortIdentity,
4041
WorkflowContext
4142
}
4243
import org.apache.texera.amber.engine.architecture.coordinator.{
@@ -48,11 +49,17 @@ import org.apache.texera.amber.engine.architecture.rpc.controlcommands.WorkflowR
4849
import org.apache.texera.amber.engine.architecture.rpc.controlreturns.EmptyReturn
4950
import org.apache.texera.amber.engine.architecture.rpc.coordinatorservice.CoordinatorServiceFs2Grpc
5051
import org.apache.texera.amber.engine.common.client.AmberClient
52+
import org.apache.texera.amber.operator.filter.{
53+
ComparisonType,
54+
FilterPredicate,
55+
SpecializedFilterOpDesc
56+
}
5157
import org.apache.texera.amber.operator.limit.LimitOpDesc
58+
import org.apache.texera.amber.operator.loop.{LoopEndOpDesc, LoopStartOpDesc}
5259
import org.apache.texera.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
5360
import org.apache.texera.amber.operator.{LogicalOp, StateTransferFunc}
5461
import org.apache.texera.amber.util.JSONUtils.objectMapper
55-
import org.apache.texera.common.compiler.model.LogicalPlan
62+
import org.apache.texera.common.compiler.model.{LogicalLink, LogicalPlan}
5663
import org.apache.texera.web.model.websocket.event.TexeraWebSocketEvent
5764
import org.apache.texera.web.model.websocket.request.ModifyLogicRequest
5865
import org.apache.texera.web.model.websocket.response.{
@@ -470,6 +477,99 @@ class ExecutionReconfigurationServiceSpec
470477
queued.map(_._1.id) shouldBe List(alreadyQueued.id)
471478
}
472479

480+
// ---------------------------------------------------------------------------
481+
// An operator that refers to loop variables
482+
// ---------------------------------------------------------------------------
483+
484+
private def loopStartOp(): LoopStartOpDesc = {
485+
val op = new LoopStartOpDesc
486+
op.initialization = "i = 0"
487+
op.output = "table.iloc[i]"
488+
op
489+
}
490+
491+
private def loopEndOp(): LoopEndOpDesc = {
492+
val op = new LoopEndOpDesc
493+
op.update = "i += 1"
494+
op.condition = "i < len(table)"
495+
op
496+
}
497+
498+
/** A service over a workflow whose plan is LoopStart -> `deployedOp` -> LoopEnd. */
499+
private def insideLoopBlock(deployedOp: LogicalOp): (ExecutionStateStore, WorkflowOnlyService) = {
500+
val start = loopStartOp()
501+
val end = loopEndOp()
502+
def link(from: LogicalOp, to: LogicalOp): LogicalLink =
503+
LogicalLink(from.operatorIdentifier, PortIdentity(), to.operatorIdentifier, PortIdentity())
504+
val workflow = Workflow(
505+
new WorkflowContext(workflowId = testWorkflowId, executionId = testExecutionId),
506+
LogicalPlan(
507+
List(start, deployedOp, end),
508+
List(link(start, deployedOp), link(deployedOp, end))
509+
),
510+
PhysicalPlan(Set.empty, Set.empty)
511+
)
512+
val stateStore = new ExecutionStateStore
513+
(stateStore, new WorkflowOnlyService(stateStore, workflow))
514+
}
515+
516+
private def filterOp(opId: String, value: String): SpecializedFilterOpDesc = {
517+
val op = new SpecializedFilterOpDesc
518+
op.setOperatorId(opId)
519+
op.predicates = List(new FilterPredicate("line", ComparisonType.EQUAL_TO, value))
520+
op
521+
}
522+
523+
private def refused(name: String): String =
524+
s"$name refers to loop variables, so it can't be changed while the workflow runs."
525+
526+
"modifyOperatorLogic" should
527+
"refuse to modify an operator whose deployed setting refers to a loop variable" in {
528+
// The references the compiler left on the deployed operator, as it does inside a loop block.
529+
val deployedOp = mkLimitOp("limit-a", 0)
530+
deployedOp.stateReferences = Map("/limit" -> "n")
531+
val (stateStore, service) = insideLoopBlock(deployedOp)
532+
533+
service.modifyOperatorLogic(ModifyLogicRequest(mkLimitOp("limit-a", 7))) shouldBe
534+
ModifyLogicResponse("limit-a", isValid = false, refused("Limit"))
535+
stateStore.reconfigurationStore.getState.unscheduledReconfigurations shouldBe empty
536+
}
537+
538+
it should "refuse an edit that makes an operator inside a loop block refer to a loop variable" in {
539+
val (typedStore, typed) = insideLoopBlock(mkLimitOp("limit-a", 1))
540+
// As the frontend sends it: "$n" in the Int property.
541+
val typedEdit = objectMapper.readValue(
542+
"""{"operatorID":"limit-a","limit":"$n","operatorType":"Limit"}""",
543+
classOf[LogicalOp]
544+
)
545+
typed.modifyOperatorLogic(ModifyLogicRequest(typedEdit)) shouldBe
546+
ModifyLogicResponse("limit-a", isValid = false, refused("Limit"))
547+
typedStore.reconfigurationStore.getState.unscheduledReconfigurations shouldBe empty
548+
549+
// A whole-string "$x" refers to a loop variable inside a loop block; the compiler never saw it.
550+
val (literalStore, literal) = insideLoopBlock(filterOp("filter-a", "Asia"))
551+
literal.modifyOperatorLogic(ModifyLogicRequest(filterOp("filter-a", "$x"))) shouldBe
552+
ModifyLogicResponse("filter-a", isValid = false, refused("Filter"))
553+
literalStore.reconfigurationStore.getState.unscheduledReconfigurations shouldBe empty
554+
}
555+
556+
it should "still modify an operator inside a loop block that refers to no loop variable, and a '$x' literal outside every block" in {
557+
val (insideStore, inside) = insideLoopBlock(mkLimitOp("limit-a", 1))
558+
inside.modifyOperatorLogic(ModifyLogicRequest(mkLimitOp("limit-a", 7))) shouldBe
559+
ModifyLogicResponse("limit-a", isValid = true, "")
560+
insideStore.reconfigurationStore.getState.unscheduledReconfigurations.map {
561+
case (op, _) => limitOf(op)
562+
} shouldBe List(7)
563+
564+
// Outside every block "$x" is the literal it is on main.
565+
val outsideStore = new ExecutionStateStore
566+
val outside =
567+
new WorkflowOnlyService(outsideStore, mkWorkflow(List(filterOp("filter-a", "Asia"))))
568+
outside.modifyOperatorLogic(ModifyLogicRequest(filterOp("filter-a", "$x"))) shouldBe
569+
ModifyLogicResponse("filter-a", isValid = true, "")
570+
outsideStore.reconfigurationStore.getState.unscheduledReconfigurations should have size 1
571+
}
572+
473573
"performReconfigurationOnResume" should
474574
"return without dispatching when no reconfigurations are pending" in {
475575
val stateStore = new ExecutionStateStore()

‎common/workflow-compiler/src/main/scala/org/apache/texera/common/compiler/WorkflowCompiler.scala‎

Lines changed: 30 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,10 @@ import org.apache.texera.common.compiler.WorkflowCompiler.{
3030
}
3131
import org.apache.texera.common.compiler.model.{LogicalPlan, LogicalPlanPojo}
3232
import org.apache.texera.amber.core.executor.OpExecWithCode
33-
import org.apache.texera.amber.core.state.StateReferencing.literalReferences
33+
import org.apache.texera.amber.core.state.StateReferencing.{
34+
literalReferences,
35+
noLoopVariableReferences
36+
}
3437
import org.apache.texera.amber.core.tuple.Schema
3538
import org.apache.texera.amber.core.virtualidentity.OperatorIdentity
3639
import org.apache.texera.amber.core.workflow.{
@@ -82,16 +85,19 @@ object WorkflowCompiler {
8285
/**
8386
* Makes `logicalOp`'s `stateReferences` sidecar final, before its physical plan serializes it
8487
* into the executor's descString. Inside a loop block it adds every whole-string `$name` value
85-
* of the operator's JSON (`StateReferencing.literalReferences`). Outside every block a string
86-
* such as "$AAPL" stays a literal, and a typed placeholder (the user wrote "$n" for an Int),
87-
* which nothing would bind, is an error; it stays in the sidecar, so a recompile reports it too.
88+
* of the operator's JSON (`StateReferencing.literalReferences`), and an entry in a property that
89+
* cannot hold a loop variable is an error (`referencesInNoLoopVariableProperties`). Outside
90+
* every block a string such as "$AAPL" stays a literal, and a typed placeholder (the user wrote
91+
* "$n" for an Int), which nothing would bind, is an error; it stays in the sidecar, so a
92+
* recompile reports it too.
8893
*/
8994
def normalizeStateReferences(logicalOp: LogicalOp, plan: LogicalPlan): Option[Throwable] =
9095
if (plan.operatorsInsideLoopBlocks.contains(logicalOp.operatorIdentifier)) {
9196
logicalOp.stateReferences =
9297
literalReferences(objectMapper.valueToTree[ObjectNode](logicalOp)) ++
9398
logicalOp.stateReferences
94-
inputsAheadOfLoopState(logicalOp, plan)
99+
referencesInNoLoopVariableProperties(logicalOp)
100+
.orElse(inputsAheadOfLoopState(logicalOp, plan))
95101
} else {
96102
Option.when(logicalOp.stateReferences.nonEmpty) {
97103
new IllegalArgumentException(
@@ -101,6 +107,25 @@ object WorkflowCompiler {
101107
}
102108
}
103109

110+
/**
111+
* The loop state writes a loop variable into the operator's setting only once the workflow runs,
112+
* but a property marked `NoLoopVariable` -- the name or type of an output column, or what an
113+
* input is partitioned on -- has been built into the plan by then: from the placeholder. A
114+
* Projection alias "$a" would output a column named "$a", null in every row; a SortPartitions
115+
* domain would stay 0 to 0. A reference there is an error instead.
116+
*/
117+
private def referencesInNoLoopVariableProperties(logicalOp: LogicalOp): Option[Throwable] = {
118+
val references = noLoopVariableReferences(logicalOp)
119+
Option.when(references.nonEmpty) {
120+
val those = if (references.size == 1) "that property" else "those properties"
121+
new IllegalArgumentException(
122+
s"${logicalOp.operatorInfo.userFriendlyName} cannot refer to loop variables " +
123+
s"(${formatReferences(references)}): its output schema or partitioning is built from " +
124+
s"$those when the workflow is compiled, before the loop runs"
125+
)
126+
}
127+
}
128+
104129
/**
105130
* The first tuple fails on a loop variable its state has not written yet, and the states of
106131
* the blocks around the operator come ahead of the tuples only on links from inside all of

0 commit comments

Comments
 (0)