Skip to content

Commit 1e79ea1

Browse files
madhur13490madhur13490
authored
[LoopUnroll] Support parallel reductions for minmax (#182473)
This patch * Supports parallel reductions for min/max operations in LoopUnroller. * Adds relevant test (including intrinsics). * Renames flag -unroll-add-parallel-reduction to -unroll-parallel-reduction. * Relaxes check in IVDescriptors.cpp (`getMinMaxRecurrence`) to handle out-of-loop uses. Planning to take support for vector types in the next patch.
1 parent 4044ea3 commit 1e79ea1

2 files changed

Lines changed: 913 additions & 7 deletions

File tree

‎llvm/lib/Transforms/Utils/LoopUnroll.cpp‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1534,9 +1534,12 @@ llvm::UnrollLoop(Loop *L, UnrollLoopOptions ULO, LoopInfo *LI,
15341534
Builder.setFastMathFlags(Reductions.begin()->second.getFastMathFlags());
15351535
RecurKind RK = Reductions.begin()->second.getRecurrenceKind();
15361536
for (Instruction *RdxPart : drop_begin(PartialReductions)) {
1537-
RdxResult = Builder.CreateBinOp(
1538-
(Instruction::BinaryOps)RecurrenceDescriptor::getOpcode(RK),
1539-
RdxPart, RdxResult, "bin.rdx");
1537+
if (RecurrenceDescriptor::isMinMaxRecurrenceKind(RK))
1538+
RdxResult = createMinMaxOp(Builder, RK, RdxResult, RdxPart);
1539+
else
1540+
RdxResult = Builder.CreateBinOp(
1541+
(Instruction::BinaryOps)RecurrenceDescriptor::getOpcode(RK),
1542+
RdxPart, RdxResult, "bin.rdx");
15401543
}
15411544
NeedToFixLCSSA = true;
15421545
for (Instruction *RdxPart : PartialReductions)
@@ -1713,11 +1716,9 @@ llvm::canParallelizeReductionWhenUnrolling(PHINode &Phi, Loop *L,
17131716
return std::nullopt;
17141717
RecurKind RK = RdxDesc.getRecurrenceKind();
17151718
// Skip unsupported reductions.
1716-
// TODO: Handle additional reductions, including FP and min-max
1717-
// reductions.
1719+
// TODO: Handle any-of and find-last reductions.
17181720
if (RecurrenceDescriptor::isAnyOfRecurrenceKind(RK) ||
1719-
RecurrenceDescriptor::isFindRecurrenceKind(RK) ||
1720-
RecurrenceDescriptor::isMinMaxRecurrenceKind(RK))
1721+
RecurrenceDescriptor::isFindRecurrenceKind(RK))
17211722
return std::nullopt;
17221723

17231724
if (RdxDesc.hasExactFPMath())

0 commit comments

Comments
 (0)