On a local test:

src/
  p1/foo/p1/
    tests/
      test_p1.py
    __init__.py
    test_p1.py
  p2/foo/p2/
    __init__.py
    test_p2.py

Without the fix:

FAILED src/p1/foo/p1/test_p1.py::test_p1 - AssertionError
FAILED src/p1/foo/p1/tests/test_p1.py::test_p1 - AssertionError
FAILED src/p2/foo/p2/test_p2.py::test_p2 - AssertionError

With the fix:

FAILED src/p1/foo/p1/test_p1.py::test_p1 - assert False is True
 +  where False = p1()
FAILED src/p1/foo/p1/tests/test_p1.py::test_p1 - assert False is True
 +  where False = p1()
FAILED src/p2/foo/p2/test_p2.py::test_p2 - assert False is True
 +  where False = p2()

Thanks @isra17 for the proof of concept. I do think at the very least we're going to want a regression test to capture expectation that was missed prior to your patch. Is that something that you'd be interested in developing?

Thanks @isra17 for the proof of concept. I do think at the very least we're going to want a regression test to capture expectation that was missed prior to your patch. Is that something that you'd be interested in developing?

Sure, I can work on this next week 👍

I can confirm that this un-breaks assertion re-writing for Matplotlib with an editable install.

@jaraco @isra17 I very much like this PR, because it would unblock us. You said that a regression test is needed: Could you give me some more detailed instructions how such a test could/should look like, and where it should be located? Perhaps I would manage to draft such a test. Though, I have to admit that I have never contributed to pytest before.

I'd model a test after some of the existing tests for importlib imports. Instead of checking that assert rewrites work, which would be much more involved, figure out what is the condition where import_path misbehaves (maybe inspired by the minimal repro in #12044 (comment)) and what that misbehavior looks like from the perspective of import_path. Then assert the correct behavior and it should fail. If you can do that on main and show the failure, then cherry-pick onto this branch and it succeeds, then you're golden.