Skip to content

Fix dense_to_sparse backward pass for three-dimensional input - #10758

Open
JOhnsonKC201 wants to merge 3 commits into
pyg-team:masterfrom
JOhnsonKC201:fix/dense-to-sparse-backward
Open

JOhnsonKC201 wants to merge 3 commits into
pyg-team:masterfrom
JOhnsonKC201:fix/dense-to-sparse-backward

Conversation

@JOhnsonKC201

Copy link
Copy Markdown

Fixes #10318.

Problem

dense_to_sparse breaks autograd when the adjacency matrix is three-dimensional:

adj = torch.rand(1, 5, 5, requires_grad=True)
edge_index, edge_attr = dense_to_sparse(adj)
edge_attr.sum().backward()
# RuntimeError: one of the variables needed for gradient computation has been
# modified by an inplace operation: [torch.LongTensor [25]] is at version 2;
# expected version 0 instead.

The equivalent 2D call works fine, which is the clue.

Cause

In the 3D branch of torch_geometric/utils/sparse.py:

edge_index = flatten_adj.nonzero().t()
edge_attr = flatten_adj[edge_index[0], edge_index[1]]   # saves the index tensors
...
edge_index[1] += offset[edge_index[0]]                  # mutates them in place

The advanced indexing on the second line saves edge_index[0] and
edge_index[1] for the index backward. The in-place += then bumps their
version counter, so autograd refuses to run backward. The 2D branch never
mutates edge_index, which is exactly why only the 3D path is affected.

Fix

Build the shifted edge_index out-of-place. One line, no API change, no
behavioural change.

I checked that the forward output is unchanged rather than assuming it: for a
seeded 3 x 4 x 4 input, both edge_index and edge_attr are identical to
main on the unmasked path and on the mask= path.

Test plan

  • New test_dense_to_sparse_backward in test/utils/test_sparse.py,
    parametrised over mask=None and a real mask. Both cases fail on main with
    the RuntimeError above and pass with this change.
  • pytest test/utils/test_sparse.py: 20 passed.
  • flake8, isort --check-only and yapf --diff are clean on both changed
    files.

@JOhnsonKC201

Copy link
Copy Markdown
Author

@akihironitta Gentle ping on this one and #10794: both still merge cleanly into master with pre-commit passing, but their Testing and Linting workflows are waiting on maintainer approval to run. Could you approve the workflows or take a look when you have a moment?

@JOhnsonKC201

Copy link
Copy Markdown
Author

@puririshi98 would you be able to take a look at this one? It fixes #10318 with a one-line change (the 3D branch of dense_to_sparse mutated edge_index in place after indexing had saved it for backward). It still merges cleanly into master, and the new test fails on master and passes with the fix.

The Testing and Linting workflows have never run because they are waiting on approval for a first-time contributor, so approving them would be the first step.

For context, #10842 was opened on 10-02 for the same issue with a similar fix. I'm happy for whichever one you prefer to go in.

For 3D input, edge_attr is gathered with flatten_adj[edge_index[0],
edge_index[1]], which saves those index tensors for the backward pass.
The following line then shifted the column indices with an in-place +=,
bumping their version counter, so calling backward on edge_attr raised
a RuntimeError about a variable modified by an inplace operation. The
2D branch has no such mutation, which is why only 3D was affected.

Build the shifted edge_index out-of-place instead. Forward output is
unchanged for both the masked and unmasked paths.
@JOhnsonKC201
JOhnsonKC201 force-pushed the fix/dense-to-sparse-backward branch from d68e3f8 to 1f903ff Compare October 23, 2026 03:35

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dense_to_sparse causes in-place operation error with 3D adjacency tensors in backward pass

1 participant