[MemCpyOpt] Use EarliestEscapeInfo by nikic · Pull Request #110280 · llvm/llvm-project

nikic · 2024-09-27T15:03:28Z

Pass EarliestEscapeInfo to BatchAA in MemCpyOpt. This allows memcpy elimination in cases where one of the involved pointers is captured after the relevant memcpy/call.

llvmbot · 2024-09-27T15:04:05Z

@llvm/pr-subscribers-llvm-transforms

Author: Nikita Popov (nikic)

Changes

Pass EarliestEscapeInfo to BatchAA in MemCpyOpt. This allows memcpy elimination in cases where one of the involved pointers is captured after the relevant memcpy/call.

Full diff: https://github.com/llvm/llvm-project/pull/110280.diff

3 Files Affected:

(modified) llvm/include/llvm/Transforms/Scalar/MemCpyOptimizer.h (+2)
(modified) llvm/lib/Transforms/Scalar/MemCpyOptimizer.cpp (+7-4)
(modified) llvm/test/Transforms/MemCpyOpt/memcpy.ll (+4-10)

diff --git a/llvm/include/llvm/Transforms/Scalar/MemCpyOptimizer.h b/llvm/include/llvm/Transforms/Scalar/MemCpyOptimizer.h
index 8ed03d7f3ddbff..e5f78ac2286833 100644
--- a/llvm/include/llvm/Transforms/Scalar/MemCpyOptimizer.h
+++ b/llvm/include/llvm/Transforms/Scalar/MemCpyOptimizer.h
@@ -26,6 +26,7 @@ class AssumptionCache;
 class CallBase;
 class CallInst;
 class DominatorTree;
+class EarliestEscapeInfo;
 class Function;
 class Instruction;
 class LoadInst;
@@ -48,6 +49,7 @@ class MemCpyOptPass : public PassInfoMixin<MemCpyOptPass> {
   PostDominatorTree *PDT = nullptr;
   MemorySSA *MSSA = nullptr;
   MemorySSAUpdater *MSSAU = nullptr;
+  EarliestEscapeInfo *EEI = nullptr;
 
 public:
   MemCpyOptPass() = default;
diff --git a/llvm/lib/Transforms/Scalar/MemCpyOptimizer.cpp b/llvm/lib/Transforms/Scalar/MemCpyOptimizer.cpp
index e9f212fb482c39..5f7dac14c1deeb 100644
--- a/llvm/lib/Transforms/Scalar/MemCpyOptimizer.cpp
+++ b/llvm/lib/Transforms/Scalar/MemCpyOptimizer.cpp
@@ -284,6 +284,7 @@ static bool mayBeVisibleThroughUnwinding(Value *V, Instruction *Start,
 
 void MemCpyOptPass::eraseInstruction(Instruction *I) {
   MSSAU->removeMemoryAccess(I);
+  EEI->removeInstruction(I);
   I->eraseFromParent();
 }
 
@@ -638,7 +639,7 @@ bool MemCpyOptPass::processStoreOfLoad(StoreInst *SI, LoadInst *LI,
   if (!LI->isSimple() || !LI->hasOneUse() || LI->getParent() != SI->getParent())
     return false;
 
-  BatchAAResults BAA(*AA);
+  BatchAAResults BAA(*AA, EEI);
   auto *T = LI->getType();
   // Don't introduce calls to memcpy/memmove intrinsics out of thin air if
   // the corresponding libcalls are not available.
@@ -1751,7 +1752,7 @@ bool MemCpyOptPass::processMemCpy(MemCpyInst *M, BasicBlock::iterator &BBI) {
         return true;
       }
 
-  BatchAAResults BAA(*AA);
+  BatchAAResults BAA(*AA, EEI);
   // FIXME: Not using getClobberingMemoryAccess() here due to PR54682.
   MemoryAccess *AnyClobber = MA->getDefiningAccess();
   MemoryLocation DestLoc = MemoryLocation::getForDest(M);
@@ -1876,7 +1877,7 @@ bool MemCpyOptPass::processByValArgument(CallBase &CB, unsigned ArgNo) {
   if (!CallAccess)
     return false;
   MemCpyInst *MDep = nullptr;
-  BatchAAResults BAA(*AA);
+  BatchAAResults BAA(*AA, EEI);
   MemoryAccess *Clobber = MSSA->getWalker()->getClobberingMemoryAccess(
       CallAccess->getDefiningAccess(), Loc, BAA);
   if (auto *MD = dyn_cast<MemoryDef>(Clobber))
@@ -1949,7 +1950,7 @@ bool MemCpyOptPass::processByValArgument(CallBase &CB, unsigned ArgNo) {
 /// 4. The memcpy src is not modified during the call. (ModRef check shows no
 /// Mod.)
 bool MemCpyOptPass::processImmutArgument(CallBase &CB, unsigned ArgNo) {
-  BatchAAResults BAA(*AA);
+  BatchAAResults BAA(*AA, EEI);
   Value *ImmutArg = CB.getArgOperand(ArgNo);
 
   // 1. Ensure passed argument is immutable during call.
@@ -2117,6 +2118,8 @@ bool MemCpyOptPass::runImpl(Function &F, TargetLibraryInfo *TLI_,
   MSSA = MSSA_;
   MemorySSAUpdater MSSAU_(MSSA_);
   MSSAU = &MSSAU_;
+  EarliestEscapeInfo EEI_(*DT);
+  EEI = &EEI_;
 
   while (true) {
     if (!iterateOnFunction(F))
diff --git a/llvm/test/Transforms/MemCpyOpt/memcpy.ll b/llvm/test/Transforms/MemCpyOpt/memcpy.ll
index 0114b9c2443582..39b90adc74ef38 100644
--- a/llvm/test/Transforms/MemCpyOpt/memcpy.ll
+++ b/llvm/test/Transforms/MemCpyOpt/memcpy.ll
@@ -837,7 +837,7 @@ define void @memcpy_memcpy_escape_after1(ptr noalias %P, ptr noalias %Q) {
 ; CHECK-NEXT:    [[MEMTMP:%.*]] = alloca [32 x i8], align 16
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 16 [[MEMTMP]], ptr align 16 [[P:%.*]], i32 32, i1 false)
 ; CHECK-NEXT:    call void @do_something()
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 16 [[Q:%.*]], ptr align 16 [[MEMTMP]], i32 32, i1 false)
+; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 16 [[Q:%.*]], ptr align 16 [[P]], i32 32, i1 false)
 ; CHECK-NEXT:    call void @capture(ptr [[MEMTMP]])
 ; CHECK-NEXT:    ret void
 ;
@@ -851,10 +851,8 @@ define void @memcpy_memcpy_escape_after1(ptr noalias %P, ptr noalias %Q) {
 
 define void @memcpy_memcpy_escape_after2(ptr noalias %P, ptr noalias %Q) {
 ; CHECK-LABEL: @memcpy_memcpy_escape_after2(
-; CHECK-NEXT:    [[MEMTMP:%.*]] = alloca [32 x i8], align 16
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 16 [[MEMTMP]], ptr align 16 [[P:%.*]], i32 32, i1 false)
 ; CHECK-NEXT:    call void @do_something()
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 16 [[Q:%.*]], ptr align 16 [[MEMTMP]], i32 32, i1 false)
+; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 16 [[Q:%.*]], ptr align 16 [[P:%.*]], i32 32, i1 false)
 ; CHECK-NEXT:    call void @capture(ptr [[P]])
 ; CHECK-NEXT:    ret void
 ;
@@ -868,10 +866,8 @@ define void @memcpy_memcpy_escape_after2(ptr noalias %P, ptr noalias %Q) {
 
 define void @memcpy_byval_escape_after(ptr noalias %P) {
 ; CHECK-LABEL: @memcpy_byval_escape_after(
-; CHECK-NEXT:    [[A:%.*]] = alloca [8 x i8], align 1
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i64(ptr align 4 [[A]], ptr align 4 [[P:%.*]], i64 8, i1 false)
 ; CHECK-NEXT:    call void @do_something()
-; CHECK-NEXT:    call void @test4a(ptr byval(i8) align 1 [[A]])
+; CHECK-NEXT:    call void @test4a(ptr byval(i8) align 1 [[P:%.*]])
 ; CHECK-NEXT:    call void @capture(ptr [[P]])
 ; CHECK-NEXT:    ret void
 ;
@@ -885,10 +881,8 @@ define void @memcpy_byval_escape_after(ptr noalias %P) {
 
 define void @memcpy_immut_escape_after(ptr align 4 noalias %val) {
 ; CHECK-LABEL: @memcpy_immut_escape_after(
-; CHECK-NEXT:    [[VAL1:%.*]] = alloca i8, align 4
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i64(ptr align 4 [[VAL1]], ptr align 4 [[VAL:%.*]], i64 1, i1 false)
 ; CHECK-NEXT:    call void @do_something()
-; CHECK-NEXT:    call void @f(ptr noalias nocapture readonly align 4 [[VAL1]])
+; CHECK-NEXT:    call void @f(ptr noalias nocapture readonly align 4 [[VAL:%.*]])
 ; CHECK-NEXT:    call void @capture(ptr [[VAL]])
 ; CHECK-NEXT:    ret void
 ;

dianqk

LGTM, thanks!
This context is https://rust-lang.zulipchat.com/#narrow/stream/187780-t-compiler.2Fwg-llvm/topic/nocapture.20and.20allocation.20elimination.

dtcxzyw

LGTM. Thank you!

dtcxzyw · 2024-09-28T07:29:14Z

llvm/lib/Transforms/Scalar/MemCpyOptimizer.cpp


 void MemCpyOptPass::eraseInstruction(Instruction *I) {
  MSSAU->removeMemoryAccess(I);
+  EEI->removeInstruction(I);


Should we call MemCpyOptPass::eraseInstruction in

llvm-project/llvm/lib/Transforms/Scalar/MemCpyOptimizer.cpp

Line 1157 in a94060c

NewCopySource->eraseFromParent();

? NewCopySource may be deleted after a BAA query.

Nice catch, fixed!

After llvm#110280 we use EarliestEscapeInfo, so callCapturesBefore() is mostly redundant. It can provide better results in some cases because EEI does an approximation across all captures. I hope that it's good enough in practice, as this is the last remaining user of callCapturesBefore(), so we'll be able to remove this API entirely.

[MemCpyOpt] Use EarliestEscapeInfo

9736bcf

Pass EarliestEscapeInfo to BatchAA in MemCpyOpt. This allows memcpy elimination in cases where one of the involved pointers is captured after the relevant memcpy/call.

nikic requested review from dianqk, dtcxzyw and fhahn September 27, 2024 15:03

llvmbot added the llvm:transforms label Sep 27, 2024

This was referenced Sep 27, 2024

Task submission dtcxzyw/llvm-opt-benchmark#1312

Open

pre-commit: PR110280 dtcxzyw/llvm-opt-benchmark#1387

Closed

dianqk approved these changes Sep 27, 2024

View reviewed changes

dtcxzyw approved these changes Sep 28, 2024

View reviewed changes

Use eraseInstruction()

f398017

the8472 mentioned this pull request Sep 28, 2024

avoid phi node for pointers flowing into Vec appends rust-lang/rust#130998

Merged

nikic merged commit f5c02dd into llvm:main Sep 30, 2024

nikic deleted the memcpyopt-eei branch September 30, 2024 07:35

nikic mentioned this pull request Sep 30, 2024

[MemCpyOpt] Stop using callCapturesBefore() #110484

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

[MemCpyOpt] Use EarliestEscapeInfo#110280

[MemCpyOpt] Use EarliestEscapeInfo#110280
nikic merged 2 commits intollvm:mainfrom
nikic:memcpyopt-eei

nikic commented Sep 27, 2024

Uh oh!

llvmbot commented Sep 27, 2024

Uh oh!

dianqk left a comment

Uh oh!

dtcxzyw left a comment

Uh oh!

dtcxzyw Sep 28, 2024

Uh oh!

nikic Sep 28, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Comments

Conversation

nikic commented Sep 27, 2024

Uh oh!

llvmbot commented Sep 27, 2024

Uh oh!

dianqk left a comment

Choose a reason for hiding this comment

Uh oh!

dtcxzyw left a comment

Choose a reason for hiding this comment

Uh oh!

dtcxzyw Sep 28, 2024

Choose a reason for hiding this comment

Uh oh!

nikic Sep 28, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants