Skip to content

Fix Enzyme autodiff ICE by anchoring metadata markers in core#155059

Open
purahan wants to merge 1 commit intorust-lang:mainfrom
purahan:fix-enzyme-lto-globals
Open

Fix Enzyme autodiff ICE by anchoring metadata markers in core#155059
purahan wants to merge 1 commit intorust-lang:mainfrom
purahan:fix-enzyme-lto-globals

Conversation

@purahan
Copy link
Copy Markdown

@purahan purahan commented Apr 9, 2026

Fixes a crash where Enzyme fails to compute derivatives when compiling with LTO. Previously, global metadata markers (like enzyme_dup) were declared in the LLVM backend (rustc_codegen_llvm). However, LLVM's Link-Time Optimization aggressively stripped their initializers, causing Enzyme to misinterpret them as active external variables rather than constant structural markers.

This fix resolves the issue by defining these structural anchors directly within the core library as #[no_mangle] statics. By anchoring them in the standard library, they are natively protected from LTO stripping, ensuring Enzyme correctly parses them as constant metadata during derivative generation.

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Apr 9, 2026
@rustbot
Copy link
Copy Markdown
Collaborator

rustbot commented Apr 9, 2026

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: @scottmcm, libs
  • @scottmcm, libs expanded to 8 candidates
  • Random selection from Mark-Simulacrum, scottmcm

@rust-log-analyzer

This comment has been minimized.

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 11, 2026
@purahan purahan force-pushed the fix-enzyme-lto-globals branch from 52242e4 to 22bdafe Compare April 12, 2026 13:11
@rustbot rustbot added the A-run-make Area: port run-make Makefiles to rmake.rs label Apr 12, 2026
@rust-log-analyzer

This comment has been minimized.

Fixes a crash where Enzyme fails to compute derivatives when compiling with LTO.
Previously, global metadata markers (like `enzyme_dup`) were declared in the
LLVM backend (`rustc_codegen_llvm`). However, LLVM's Link-Time Optimization
aggressively stripped their initializers, causing Enzyme to misinterpret them
as active external variables rather than constant structural markers.

This fix resolves the issue by defining these structural anchors directly
within the `core` library as `#[no_mangle]` statics. By anchoring them in the
standard library, they are natively protected from LTO stripping, ensuring
Enzyme correctly parses them as constant metadata during derivative generation.
@purahan purahan force-pushed the fix-enzyme-lto-globals branch from 22bdafe to 73c60ae Compare April 13, 2026 05:23
@purahan
Copy link
Copy Markdown
Author

purahan commented Apr 13, 2026

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 13, 2026
@ZuseZ4 ZuseZ4 added the F-autodiff `#![feature(autodiff)]` label Apr 17, 2026
@ZuseZ4
Copy link
Copy Markdown
Member

ZuseZ4 commented Apr 17, 2026

r? me

Hi! Do you have any tests or a reproducer which was broken and which this change is fixing?

@rustbot rustbot assigned ZuseZ4 and unassigned Mark-Simulacrum Apr 17, 2026
@purahan
Copy link
Copy Markdown
Author

purahan commented Apr 18, 2026

Hey @ZuseZ4, thanks for reviewing!

Yeah, the ICE triggers specifically when combining the autodiff feature with Link-Time Optimization (-C lto=fat).

Here’s a minimal reproducer:

  #![feature(autodiff)]
  use std::autodiff::*;
  
  #[autodiff_reverse(df, Duplicated, Duplicated)]
  fn f(x: &[f64; 2], y: &mut f64) {
      *y = x[0] * x[0] + x[1] * x[0];
  }
  
  fn main() {
      let x = [2.0, 2.0];
      let mut bx = [0.0, 0.0]; 
      let mut y = 0.0;
      let mut by = 1.0; 
  
      df(&x, &mut bx, &mut y, &mut by);
      assert_eq!(bx, [6.0, 2.0]);
  }

If you compile that with rustc --edition 2021 -Z autodiff=Enable -C lto=fat test_autodiff.rs on the codebase before this PR, you get this Enzyme backend panic:

error: <unknown>:0:0: in function preprocess__RNvCsim5ttFnN1cV_13test_autodiff2df void (ptr, ptr, ptr, ptr): Enzyme: cannot compute with global variable that doesn't have marked shadow global
@enzyme_dup = external global ptr

Basically, because those dummy metadata globals were only declared in the C++ backend, LTO was aggressively stripping their initializers out as dead code. Moving them directly into core natively protects them from LTO, so Enzyme can actually read them.

Let me know if you want me to drop this reproducer into tests/ui or tests/run-make as an official test case, happy to push a commit for that!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-run-make Area: port run-make Makefiles to rmake.rs F-autodiff `#![feature(autodiff)]` S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants