Audit Comet logical-link repair across AQE replanning #6034

What is the problem the feature request solves?

CometExecRule repairs links on CometExec, CometShuffleExchangeExec, and CometBroadcastExchangeExec using originalPlan.logicalLink, or clears both logical tags when that link is absent. #5482 / #5483 establish that originalPlan is not always authoritative: Spark can reuse an existing physical root and assign a direct link to the current LogicalQueryStage.

#5483 preserves that direct link on CometExec. This issue tracks the separate question raised in the review: which remaining repairs are necessary, and can link ownership be made explicit or repair narrowed without breaking AQE?

Describe the potential solution

Audit initial conversion, repeated AQE replanning, and exchange/subquery creation to document when the original plan, the current physical node, or Spark's temporary stage link is authoritative. Distinguish fresh nodes from reused physical nodes and direct tags from inherited tags. Decide from that evidence whether to retain the current repair, narrow it, or move necessary assignments to node creation.

Any change should preserve:

• The current direct LogicalQueryStage identity across repeated replanning (CometExecRule overwrites direct AQE LogicalQueryStage links during replanning #5482).
• Necessary restoration/clearing for ordinary and inherited links, including the empty shuffle-link behavior from CometShuffleExchangeExec logical link is different to Spark ShuffleExchangeExec #323.
• Ordinary shuffle/broadcast stage wrapping and DPP subqueries whose adaptive-plan root is a CometBroadcastExchangeExec. The DPP investigation found an unchanged, equal-cost replan that Spark rejects, while retaining TEMP_LOGICAL_PLAN_TAG for new stage creation. Establish the required contract across supported Spark versions.

Back the chosen contract with focused planner tests and end-to-end AQE/DPP coverage that checks results and actual native execution. This audit does not assume all repair is removable or establish another failing exchange case.

Additional context

Keep #5482 focused on its reproduced stale-stage-link defect. The broader design question should remain tracked after #5483 closes that issue.

Current logical-link repair.