Skip to content

Comments

[flake8-async] Extend ASYNC230 to detect blocking pathlib I/O methods#19619

Closed
aidandj wants to merge 5 commits intoastral-sh:mainfrom
aidandj:aidan/async-230-pathlib-read-write
Closed

[flake8-async] Extend ASYNC230 to detect blocking pathlib I/O methods#19619
aidandj wants to merge 5 commits intoastral-sh:mainfrom
aidandj:aidan/async-230-pathlib-read-write

Conversation

@aidandj
Copy link

@aidandj aidandj commented Jul 29, 2025

Summary

Add detection for pathlib's blocking I/O methods (read_text, read_bytes, write_text, write_bytes) in async functions, beyond just the open() method. Under the hood, these methods call open, so they have the same issue that open does.

Test Plan

Added test cases to the existing ASYNC230 test suite and updated the snapshot.

Add detection for pathlib's blocking I/O methods (read_text, read_bytes,
write_text, write_bytes) in async functions, beyond just the open() method.

Signed-off-by: Aidan Jensen <[email protected]>
Copy link
Contributor

@ntBre ntBre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This change makes sense to me, but I think we should preview-gate it since the rule itself is stable.

Otherwise I just had a couple of small nits.

Comment on lines 77 to 82
if attr.as_str() != "open"
&& attr.as_str() != "read_text"
&& attr.as_str() != "read_bytes"
&& attr.as_str() != "write_text"
&& attr.as_str() != "write_bytes"
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could use matches! here:

Suggested change
if attr.as_str() != "open"
&& attr.as_str() != "read_text"
&& attr.as_str() != "read_bytes"
&& attr.as_str() != "write_text"
&& attr.as_str() != "write_bytes"
{
if !matches!(attr.as_str(), "open"
| "read_text"
| "read_bytes"
| "write_text"
| "write_bytes")
{

However, we will need two checks here for now because I think we should gate this behind preview since it's an expansion of a stable rule. You can see examples of these in preview.rs:

// https://github.com/astral-sh/ruff/pull/19100
pub(crate) const fn is_add_future_annotations_imports_enabled(settings: &LinterSettings) -> bool {
settings.preview.is_enabled()
}

and references to those functions.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the matches suggestion. Rust newbie here, still learning.

I'll look into the preview stuff.

def foo_sync():
open("")


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you revert this? It will make the snapshots a bit easier to review since it won't shift everything down :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I kept trying to revert this but my formatter kept adding it back. I'll revert it again.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries, I habitually hit my formatter binding and always have to hold back when modifying snapshots too.

@ntBre ntBre added preview Related to preview mode features rule Implementing or modifying a lint rule labels Jul 29, 2025
@ntBre ntBre changed the title [flake8-async] Extend ASYNC230 to detect blocking pathlib I/O methods [flake8-async] Extend ASYNC230 to detect blocking pathlib I/O methods Jul 29, 2025
@aidandj aidandj requested a review from ntBre July 29, 2025 19:33
@aidandj
Copy link
Author

aidandj commented Jul 29, 2025

I did a double check in the pathlib library and found touch as well calls io.open. Added that in there.

I found a pattern of enabling preview tests in another file and copied that over, if there is a better method for preview tests let me know. It would be nice to tie them to those helper methods so that when you deleted the helper method you would know to move the test out of the preview call. Maybe that could be as simple as calling the helper function in the preview test to bring attention to it.

Also, who's responsibility is it to take it out of preview? The PR author? Or the ruff team?

Signed-off-by: Aidan Jensen <[email protected]>
@MichaReiser
Copy link
Member

Also, who's responsibility is it to take it out of preview? The PR author? Or the ruff team?

We'll take it out of preview, following our versioning policy.

Is this change in line with the upstream ASYNC rule?

@ntBre
Copy link
Contributor

ntBre commented Jul 30, 2025

Is this change in line with the upstream ASYNC rule?

I should have mentioned this somewhere, but I looked into the upstream rule yesterday, and it looks like they only flag open, io.open, and io.open_code, so I concluded that we'd already diverged from upstream by handling pathlib.Path.open. But I'm certainly open to your input.

https://github.com/python-trio/flake8-async/blob/1bf61068d7cdaa25b8bdb91cbff017a785df1220/flake8_async/visitors/visitor2xx.py#L277-L278

62 | async def func() -> None:
63 | p = Path("foo").read_text() # ASYNC230
| ^^^^^^^^^^^^^^^^^^^^^ ASYNC230
|
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diagnostics here are fairly confusing as it's not clear to me where I'm calling open. Can we add a help text saying that read_text calls open internally`. We may also want to update the rule's documentation to mention the new (preview only behavior)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option would be to simplify and just remove the mention of open. And just say blocking methods.

@MichaReiser
Copy link
Member

MichaReiser commented Aug 4, 2025

I should have mentioned this somewhere, but I looked into the upstream rule yesterday, and it looks like they only flag open, io.open, and io.open_code, so I concluded that we'd already diverged from upstream by handling pathlib.Path.open. But I'm certainly open to your input.

Considering Path.open does make sense and seems less of a divergence to me than including extra methods. While the new methods are in the spirit of the rule, it does push its boundaries, e.g., a better rule name might be blocking-file-operation-in-async-function as the problem isn't just open (I suspect it's any file IO operation)?

Unlike other rules, the upstream async linter is very actively maintained. That makes it more likely that we'll run into conflicting rules.

I tried to find a related issue or PR upstream but wasn't successful. @Zac-HD: Is this something that has come up with the async rules before? Is this a change you would be open to also make in flake8-async (I'm not expecting you to make the change yourself, just if it's in line with how you designed the rules)?

@Zac-HD
Copy link

Zac-HD commented Aug 4, 2025

I think this'd be the first time ruff was ahead of flake8-async, nice!

How about we move this new logic to a new ASYNC233 code, and open an upstream issue to implement it / reserve the number? I think this is a much better experience for users updating their linter too, since they can use some temporary ignores without missing the current 230 issues.

@aidandj
Copy link
Author

aidandj commented Aug 4, 2025

I think this'd be the first time ruff was ahead of flake8-async, nice!

How about we move this new logic to a new ASYNC233 code, and open an upstream issue to implement it / reserve the number? I think this is a much better experience for users updating their linter too, since they can use some temporary ignores without missing the current 230 issues.

This seems reasonable. Let me know if there is consensus and I could take a stab at that as well

@Zac-HD
Copy link

Zac-HD commented Aug 4, 2025

Upstream issue opened!

Practically speaking, if you're interested in implementing more rules I'd pretty strongly prefer pulling more into ruff relative to moving this one in the other direction; I already mostly use ruff for speed 😁

@MichaReiser
Copy link
Member

Thanks @Zac-HD

Practically speaking, if you're interested in implementing more rules I'd pretty strongly prefer #8451 relative to moving this one in the other direction; I already mostly use ruff for speed 😁

We're interested, it's just hard to prioritize with everything else going but we could mark some of those as help wanted (CC: @ntBre wdty?)

A new code sounds good to me. But this may justify removing the exsting pathlib check from ASYNC230 (under preview)

@jakkdl
Copy link

jakkdl commented Aug 5, 2025

Thanks @Zac-HD

Practically speaking, if you're interested in implementing more rules I'd pretty strongly prefer #8451 relative to moving this one in the other direction; I already mostly use ruff for speed 😁

We're interested, it's just hard to prioritize with everything else going but we could mark some of those as help wanted (CC: @ntBre wdty?)

A new code sounds good to me. But this may justify removing the exsting pathlib check from ASYNC230 (under preview)

I don't think we have any preference between adding pathlib.Path.open to ASYNC230 vs adding all pathlib methods to an ASYNC233 in upstream, so we can simply sync to what you end up deciding to do.

@ntBre
Copy link
Contributor

ntBre commented Aug 5, 2025

We're interested, it's just hard to prioritize with everything else going but we could mark some of those as help wanted (CC: @ntBre wdty?)

Sounds good to me!

Thank y'all!

@ntBre
Copy link
Contributor

ntBre commented Jan 27, 2026

Thanks again for working on this! I'll go ahead and close this PR for now as it's gotten stale, but feel free to open another one implementing a new ASYNC233 rule as discussed above.

@ntBre ntBre closed this Jan 27, 2026
@ntBre ntBre mentioned this pull request Jan 27, 2026
38 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

preview Related to preview mode features rule Implementing or modifying a lint rule

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants