Skip to content

Remove dead CacheSourceOpExec and OpExecSource plumbing (replaced by input-port materialization readers in #3425) #6125

Closed

Description

@aglinxinyuan

Task Summary

Cache source operators were replaced by input-port materialization reader threads in #3425 (the symmetric follow-up to sink-operator removal in #3312). Since then CacheSourceOpExec has been unreachable: the scheduler no longer inserts a cache-source read op for a materialized link — it tags the downstream input port with the URI directly.

SpecialPhysicalOpFactory.newSourcePhysicalOp — the only producer of an OpExecSource init-info, which is what constructs CacheSourceOpExec in DataProcessorRPCHandlerInitializer.setupExecutor — has no production caller; its only remaining references are its own spec. This is dead code and should be removed.

Materialized-link read path, before → after:

Before (pre-#3425):  upstream OutputPort(URI) ──► [CacheSourceOpExec] ──► downstream InputPort
After  (#3425+):     upstream OutputPort(URI) ─────────────────────────► downstream InputPort(URI)

Removal scope:

Item Location Note
CacheSourceOpExec common/workflow-operator/.../source/cache/CacheSourceOpExec.scala the executor
SpecialPhysicalOpFactory (+ its spec) common/workflow-operator/.../operator/SpecialPhysicalOpFactory.scala sole producer of OpExecSource; no production caller
OpExecSource match arm + import amber/.../worker/DataProcessorRPCHandlerInitializer.scala (import L26, arm L92) unreachable branch
OpExecSource message + oneof field common/workflow-core/.../protobuf/.../executor.proto (message L42, opExecSource = 3 L51) remove; reserve field number 3 in the sealed_value oneof to prevent reuse

Notes:

  • The OpExecInitInfo oneof is an internal controller↔worker wire format (built and consumed within the same deployment; preserve_unknown_fields: false), so dropping the field is safe — just reserve 3.
  • The unit spec CacheSourceOpExecSpec was already dropped in test(workflow-operator): add unit test coverage for interval-join, and sleep #6098 (this dead-code discovery came out of that review), so nothing test-side blocks the removal.

Task Type

  • Refactor / Cleanup

Activity

  1. added theissue type on Jul 5, 2026
  2. added 2 commits that reference this issue on Jul 5, 2026
    fd3af74
    6361f9f
  3. added a commit that references this issue on Jul 5, 2026
    95684eb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions