Skip to content

Conversation

@fmeum
Copy link
Collaborator

@fmeum fmeum commented Mar 10, 2023

Fixes #17675

@fmeum
Copy link
Collaborator Author

fmeum commented Mar 10, 2023

@rickeylev Could you review?

@fmeum
Copy link
Collaborator Author

fmeum commented Mar 10, 2023

@bazel-io flag

@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Mar 10, 2023
@sgowroji sgowroji added team-Rules-Python Native rules for Python awaiting-review PR is awaiting review from an assigned reviewer labels Mar 10, 2023
@rickeylev rickeylev self-requested a review March 10, 2023 08:54
@rickeylev
Copy link
Contributor

Can we add a test for this? In src/test/shell/integration/python_stub_test.sh, we can easily plug in a no-op interpreter that just prints env, then grep for the RUNFILES_MANIFEST_FILE. I think we just need to set --build_runfile_manifests when doing the build and invoke it using bazel-bin to make the right code paths get followed.

@fmeum
Copy link
Collaborator Author

fmeum commented Mar 10, 2023

@rickeylev Started working on the test, but I can't figure out how to build an input but not an output manifest. --build_runfile_manifests does seem to control both. I could of course delete one from the output base manually, but that feels hacky.

@rickeylev
Copy link
Contributor

Hmm....yeah I don't see a way to generate only one and not the other. Maybe it's specific to certain platforms? Or maybe if...ah, what if it's running from a zip file? CreateModuleSpace() will unzip to a temp directory. The zip will contain MANIFEST, but the temp directory won't have .runfiles_manifest

In the original issue description, they manually copy the runfiles tree elsewhere and run, which is approximately the same as unzipping elsewhere.

I also think it'd be fine to just delete the .runfiles_manifest manually. It's effectively the same as the other two approaches, just with less copy+pasting of things around.

Let's just put a comment in the test about the case we're trying to reproduce; it'll look pretty funny deleting files from the output tree like that.

@ShreeM01
Copy link
Contributor

@bazel-io fork 6.2.0

@bazel-io bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Mar 10, 2023
@fmeum
Copy link
Collaborator Author

fmeum commented Mar 21, 2023

@rickeylev I tried to get this to work but noticed that I can't come up with a scenario in which the change becomes relevant: In ZIP-based builds the incorrect branch isn't reached since IsRunningFromZip() is checked earlier. With --nobuild_runfile_links and a non-ZIP-based build, Python can't run as it needs the runfiles directory structure laid out. I would need a scenario in which the runfiles directory can only be found using the manifest file, which in turn is found via argv[0] - but is there a natural case in which this is the case?

@rickeylev
Copy link
Contributor

Ah darn. Ok, well, lets just go back to the other ideas:

  • Manually deleting the {name}.runfiles_manifest file
  • The steps described in the issue: cp -r bazel-bin/foo bazel-bin/foo.runfiles /tmp/work && /tmp/work/foo

If none of that works, well, lets just omit the tests, because we can't figure out a way to test it, which is fine. That line of code is obviously wrong.

@keertk
Copy link
Member

keertk commented Apr 13, 2023

Hi @fmeum @rickeylev we're planning to start creating RCs for 6.2.0 on 4/24. Checking if we still want to include this?

@fmeum
Copy link
Collaborator Author

fmeum commented Apr 17, 2023

@rickeylev I couldn't figure out how to reproduce this situation in tests, even with these ideas. I added the returns comment and your suggestions.

@rickeylev rickeylev changed the title Set correct runfiles variable in Python stub fix(python): Set envvar for runfiles manifest, not runfiles dir, when using a manifest Apr 17, 2023
@rickeylev rickeylev added type: bug awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels Apr 17, 2023
@rickeylev
Copy link
Contributor

re: LocalDiffAwarenessIntegrationTest failure -- no idea what that is or why it would fail. Looking at the test, I don't see any mention of Python in it. I'm guessing its flaky mac test. I pressed retry.

In any case, I'd say this is fine to begin importing. Our internal test infrastructure will handle flaky tests better.

@fmeum fmeum deleted the patch-9 branch April 18, 2023 05:48
@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Apr 18, 2023
@fmeum fmeum removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Apr 18, 2023
keertk added a commit that referenced this pull request Apr 18, 2023
… using a manifest (#18133)

Unfortunately, we weren't able to find a way to reproduce the reported bug
in a test environment, but the line of code in question is obviously wrong,
so we'll just omit a test to cover this.

Fixes #17675

Closes #17722.

PiperOrigin-RevId: 525044539
Change-Id: I7e1eaa14eac1d4dabcdcf93d92720c41977b1fe2

Co-authored-by: Fabian Meumertzheim <[email protected]>
@sgowroji sgowroji removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Apr 19, 2023
fweikert pushed a commit to fweikert/bazel that referenced this pull request May 25, 2023
… using a manifest

Unfortunately, we weren't able to find a way to reproduce the reported bug
in a test environment, but the line of code in question is obviously wrong,
so we'll just omit a test to cover this.

Fixes bazelbuild#17675

Closes bazelbuild#17722.

PiperOrigin-RevId: 525044539
Change-Id: I7e1eaa14eac1d4dabcdcf93d92720c41977b1fe2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team-Rules-Python Native rules for Python type: bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python wrapper sometimes puts manifest filename into RUNFILES_DIR

6 participants