Let Replicated locus join with others(Results of writeable CTE on replicated table Join with others) #84
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have a fix for GPDB https://github.com/greenplum-db/gpdb/pull/15862 since June 19, the issue #51 exists for CBDB.
Check back to CBDB since we are open source and welcome for comments.
Most codes are same with some refinement and conflicts resolved, as we have parallel feature and change functions like cbdpath_motion_for_join().
context
Replicated locus could happen besides we Broadcast a node.
Ex: Writeable CTE on replicated table join with others.
See issue: https://github.com/greenplum-db/gpdb/issues/15860.
And all cased added in rpt_joins will cause such a failure because all that are blocked by
And while I open it, there are more issues than https://github.com/greenplum-db/gpdb/issues/15860 , we never expect to handle Replicated locus could join with others.
The pr enables all of that except OuterQuery locus.
The principle to decide join locus of Replicated with others is:
the slice does the writeable operation on replicated table
must be executed on all segments.
To be more specific, ex: Replicated join SegmentGeneral should return Replicated locus.
If return SegmentGeneral, we will allocate gang on only one segment which will insert/update/delete on only segment0 for replicated tabes. That's wrong.
Before issue 15860, we have the logic return SegmentGeneral in cdbpathlocus_join(). As we have blocked Replicated locus
in cdbpath_motion_for_join(), the only scenario is update a replicated table from values of another replicated table whose num segment is less.
in theses codes:
Wile broadcasting the d1 whose num segments is less than d2, and we return SegmentGeneral locus for the join node,
yeah, GDB it and found that the Nested Loop node locus is SegmentGeneral. That's wrong as we mentioned: allocate gang only on one segment.
I was curious why the plan is finally right until I found that adjust_modifytable_subpaths() reset the Nested Loop node locus
to Replicated locus.
That's a little hack and wried, and not work for issue 15860 and cases added in this mr.
I didn't see any good reason why Replicated join SegmentGeneral should return SegmentGeneral and force set it back to Replicated finally. And that's wrong for writeable on replicated tables from values or a non-segmentgeneral table(hashed for example)
See examples below.
DDL for examples
xxx_2_segments means define table on 2 segments of a 3 segments cluster.
Replicated join SegmentGeneral.
When locus is not ok to replicate, ex: as the nullable side of outer join, we must gather them to single.
While SegmentGeneral's num segments are different with Replicated locus, we have two ways:
It seems both are ok, but Broadcast to all segments will cost more. And as we will need a Gather Motion or Explicit Gather Motion finally, we should make as less data as possible from the end to top for planner.
Maybe we should estimate rel size of both, but it's hard to tell size of a returning statement.
table rpt_issue_15860_2_segments is defined with 2 segments of a 3 segments cluster.
Replicated join General.
Similar to SegmentGeneral, but as General could be on QD, and we must guarantee the Principle above to insert/update/delete on all segments for replicated table.
Replicate join SingleQE.
Replicate join Entry
Similar to SingleQE, and guarantee the Principle too, that is to avoid to be Motion Elided early, see codes for details.
Replicated join Partitioned
Hashed/Strewn/HashedOJ handle is similar.
When num segments are not match for Partitioned locus, we redistribute it to num segments of Replicated.
Replicated join OuterQuery.
When Replicated join OuterQuery, make Replicated to OuterQuery locus may be wrong.
OuterQuery will be finally be Broadcast or Gathered.
If it's Gathered, we will insert/update/delete only on one segment for a replicated table, that's not right.
I don't have a good idea to handle OuterQuery, left a FIXME and created the below example.
Resolve GPDB_96_MERGE_FIXME in adjust_modifytable_subpaths()
And with this change, a GPDB_96_MERGE_FIXME left by Heikki in adjust_modifytable_subpaths () can be resolved.
closes: #51
Change logs
Describe your change clearly, including what problem is being solved or what feature is being added.
If it has some breaking backward or forward compatibility, please clary.
Why are the changes needed?
Describe why the changes are necessary.
Does this PR introduce any user-facing change?
If yes, please clarify the previous behavior and the change this PR proposes.
How was this patch tested?
Please detail how the changes were tested, including manual tests and any relevant unit or integration tests.
Contributor's Checklist
Here are some reminders before you submit the pull request:
make installcheckmake -C src/test installcheck-cbdb-parallel