Skip to content

Conversation

[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 21, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/138473

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 65ca3ef with merge base 288aa87 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

rec added a commit that referenced this pull request Oct 21, 2024
* Extracted from #133492

ghstack-source-id: 7266f39
Pull Request resolved: #138473
@rec rec requested a review from albanD October 21, 2024 15:34
@rec rec added the topic: not user facing topic category label Oct 21, 2024
@Skylion007
Copy link
Collaborator

What is status variable here? Is this actually a case of missing error handling?

@rec
Copy link
Collaborator Author

rec commented Nov 22, 2024

@pytorchbot rebase

@pytorchmergebot
Copy link
Collaborator

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[ghstack-poisoned]
@pytorchmergebot
Copy link
Collaborator

Successfully rebased gh/rec/86/orig onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via ghstack checkout https://github.com/pytorch/pytorch/pull/138473)

pytorchmergebot pushed a commit that referenced this pull request Nov 22, 2024
* Extracted from #133492

ghstack-source-id: c35181b
Pull Request resolved: #138473
@rec rec mentioned this pull request Nov 30, 2024
22 tasks
x86_isa_help_builder.get_target_file_path()
)
if not os.path.isfile(output_path):
status, target_file = x86_isa_help_builder.build()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels like the status value here should be checked...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slightly unexpected results!

That line calls CppBuilder.build, which calls run_compile_command which calls _run_compile_command, which calls subprocess.check_output.

But check_output returns the contents of stdout and not a status code! (If the subprocess fails, check_output raises an exception which the code handles.)

The contents of stdout might contain any old junk and run_compile_command is only called in that one spot, so I got rid of that return value entirely.

[ghstack-poisoned]
rec added a commit that referenced this pull request Dec 2, 2024
* Extracted from #133492

ghstack-source-id: 704287a
Pull Request resolved: #138473
def build(self) -> Tuple[bytes, str]:
def build(self) -> None:
"""
It is must need a temperary directory to store object files in Windows.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Skylion007: This is a little off-topic, but I see two issues in CppBuilder.build().

The first is that the docstring is not really useful.

The second is that the temporary directory doesn't get actually get deleted if run_compile_cmd raises an exception.

If this is intended behavior, it should be documented; otherwise, the _remove_dir should be in a finally: block.

I'd be happy to fix both of these while I was here.

[ghstack-poisoned]
rec added a commit that referenced this pull request Dec 3, 2024
* Extracted from #133492

ghstack-source-id: 24fd237
Pull Request resolved: #138473
@albanD albanD requested review from eellison and removed request for albanD December 3, 2024 22:56
@eellison eellison requested review from desertfire and removed request for eellison December 3, 2024 23:03
@desertfire desertfire requested a review from xuhancn December 4, 2024 20:19
def _run_compile_cmd(cmd_line: str, cwd: str) -> None:
cmd = shlex.split(cmd_line)
try:
status = subprocess.check_output(args=cmd, cwd=cwd, stderr=subprocess.STDOUT)
Copy link
Collaborator

@xuhancn xuhancn Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I intend to keep current code. check_output need work with many OSs and many compilers. They may return different behaviors, even some error output messages. If we can't make sure we can make it better, we'd better keep current code.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xuhancn @Skylion007

status is not a status variable: it is a byte string which merges stdout and stderr. At the very least, it should be correctly named.

But this result simply isn't used at all, anywhere in the code - indeed, it's not clear how it could be used since we have no information about what the output might be, or even if that output is stable between runs of the compiler or different settings.

Keeping unused, trivial code around just in case it might be used in the future is not a good idea.

You will note that this variable has already consumed time from three different engineers, because its name gives the false impression that we had forgotten to check a status variable.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I completely agree with @rec. It doesn't make any sense to use a misleading and useless variable here. We should remove it directly. @xuhancn, if you strongly believe that checking the compilation output is necessary for robustness, please submit another PR, and we will review it carefully.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EikanWang Okay, let's merge it.

@albanD
Copy link
Collaborator

albanD commented Dec 16, 2024

cc @EikanWang I think this would need a slightly larger cleanup if we want to change the variables to actually output a proper status flag etc. Would someone on your side be able to help with that by any chance? Or is deleting all the wrong variables as done here is ok for now and we can add back proper code later when it is needed?

[ghstack-poisoned]
rec added a commit that referenced this pull request Dec 18, 2024
* Extracted from #133492

ghstack-source-id: c0c11df
Pull Request resolved: #138473
[ghstack-poisoned]
[ghstack-poisoned]
fightingand pushed a commit to fightingand/pytorch that referenced this pull request Dec 20, 2024
@EikanWang
Copy link
Collaborator

cc @EikanWang I think this would need a slightly larger cleanup if we want to change the variables to actually output a proper status flag etc. Would someone on your side be able to help with that by any chance? Or is deleting all the wrong variables as done here is ok for now and we can add back proper code later when it is needed?

Sorry for the late response. @albanD , @rec , let's remove all the wrong variables just as this PR has done.

Copy link
Collaborator

@albanD albanD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like it's ready to land, happy to help reviewing the follow up improvements if needed!

@rec
Copy link
Collaborator Author

rec commented Dec 20, 2024

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Dec 20, 2024
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

def _run_compile_cmd(cmd_line: str, cwd: str) -> None:
cmd = shlex.split(cmd_line)
try:
status = subprocess.check_output(args=cmd, cwd=cwd, stderr=subprocess.STDOUT)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EikanWang Okay, let's merge it.

@github-actions github-actions bot deleted the gh/rec/86/head branch January 20, 2025 02:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants