Skip to content

Comments

Avoid reusing nested, interpolated quotes before Python 3.12#20930

Merged
ntBre merged 2 commits intomainfrom
brent/f-string-fix
Oct 17, 2025
Merged

Avoid reusing nested, interpolated quotes before Python 3.12#20930
ntBre merged 2 commits intomainfrom
brent/f-string-fix

Conversation

@ntBre
Copy link
Contributor

@ntBre ntBre commented Oct 16, 2025 •
edited
Loading

Summary

Fixes #20774 by tracking whether an InterpolatedStringState element is nested inside of another interpolated element. This feels like kind of a naive fix, so I'm welcome to other ideas. But it resolves the problem in the issue and clears up the syntax error in the black compatibility test, without affecting many other cases.

The other affected case is actually interesting too because the input is invalid, but the previous quote selection fixed the invalid syntax:

Python 3.11.13 (main, Sep  2 2025, 14:20:25) [Clang 20.1.4 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f'{1: abcd "{'aa'}" }'  # input
  File "<stdin>", line 1
    f'{1: abcd "{'aa'}" }'
                  ^^
SyntaxError: f-string: expecting '}'
>>> f'{1: abcd "{"aa"}" }'  # old output
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier ' abcd "aa" ' for object of type 'int'
>>> f'{1: abcd "{'aa'}" }'  # new output
  File "<stdin>", line 1
    f'{1: abcd "{'aa'}" }'
                  ^^
SyntaxError: f-string: expecting '}'

We now preserve the invalid syntax in the input.

Unfortunately, this also seems to be another edge case I didn't consider in #20867 because we don't flag this as a syntax error after 0.14.1:

Shell output 

> uvx [email protected] check --ignore ALL --target-version py311 - <<EOF
f'{1: abcd "{'aa'}" }'
EOF
invalid-syntax: Cannot reuse outer quote character in f-strings on Python 3.11 (syntax was added in Python 3.12)
 --> -:1:14
  |
1 | f'{1: abcd "{'aa'}" }'
  |              ^
  |

Found 1 error.
> uvx [email protected] check --ignore ALL --target-version py311 - <<EOF
f'{1: abcd "{'aa'}" }'
EOF
All checks passed!
> uvx [email protected] -m ast <<EOF
f'{1: abcd "{'aa'}" }'
EOF
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/brent/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/ast.py", line 1752, in <module>
    main()
  File "/home/brent/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/ast.py", line 1748, in main
    tree = parse(source, args.infile.name, args.mode, type_comments=args.no_type_comments)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/brent/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<stdin>", line 1
    f'{1: abcd "{'aa'}" }'
                  ^^
SyntaxError: f-string: expecting '}'

I assumed that was the same ParseError as the one caused by f"{1:""}", but this is a nested interpolation inside of the format spec.

Test Plan

New test copied from the black compatibility test. I guess this is a duplicate now, I started working on this branch before the new black tests were imported, so I could delete the separate test in our fixtures if that's preferable.

@ntBre ntBre added bug Something isn't working formatter Related to the formatter labels Oct 16, 2025
@ntBre ntBre changed the title Brent/f string fix Avoid reusing nested, interpolated quotes before Python 3.12 Oct 16, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 16, 2025

ruff-ecosystem results

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

@ntBre ntBre marked this pull request as ready for review October 16, 2025 20:46
@ntBre ntBre requested a review from MichaReiser as a code owner October 16, 2025 20:46
@MichaReiser
Copy link
Member

MichaReiser commented Oct 17, 2025

I like the solution. It's very non-invasive, comes with little code and runtime complexity and it preserves the syntax error as is (which I think is better than trying to fixing syntax errors and than creating an even bigger mess)

@ntBre ntBre merged commit 0115fd3 into main Oct 17, 2025
39 checks passed
@ntBre ntBre deleted the brent/f-string-fix branch October 17, 2025 12:49
ntBre added a commit that referenced this pull request Oct 17, 2025
@ntBre
Catch syntax errors in nested interpolations before Python 3.12
1522d87 
Summary
--

This PR fixes the issue I added in #20867 and noticed in #20930. Cases like this
cause an error on any Python version:

```py
f"{1:""}"
```

which gave me a false sense of security before. Cases like this are still
invalid only before 3.12 and weren't flagged after the changes in #20867:

```py
f'{1: abcd "{'aa'}" }'
           # ^  reused quote
f'{1: abcd "{"\n"}" }'
            # ^  backslash
```

I didn't recognize these as nested interpolations that also need to be checked
for invalid expressions, so filtering out the whole format spec wasn't quite
right.

There's basically no code change in this PR, I just moved the existing check
from `parse_interpolated_string`, which parses the entire string, to
`parse_interpolated_element`. This kind of seems more natural anyway and avoids
having to try to recursively visit nested elements after the fact in
`parse_interpolated_string`. So viewing the diff with something like

```
git diff --color-moved --ignore-space-change --color-moved-ws=allow-indentation-change main...HEAD
```

should make this more clear.

Test Plan
--

New tests
@ntBre ntBre mentioned this pull request Oct 17, 2025
Merged
ntBre added a commit that referenced this pull request Oct 20, 2025
@ntBre
Catch syntax errors in nested interpolations before Python 3.12 (#20949)
38c074e 
Summary
--

This PR fixes the issue I added in #20867 and noticed in #20930. Cases
like this
cause an error on any Python version:

```py
f"{1:""}"
```

which gave me a false sense of security before. Cases like this are
still
invalid only before 3.12 and weren't flagged after the changes in
#20867:

```py
f'{1: abcd "{'aa'}" }'
           # ^  reused quote
f'{1: abcd "{"\n"}" }'
            # ^  backslash
```

I didn't recognize these as nested interpolations that also need to be
checked
for invalid expressions, so filtering out the whole format spec wasn't
quite
right. And `elements.interpolations()` only iterates over the outermost 
interpolations, not the nested ones.

There's basically no code change in this PR, I just moved the existing
check
from `parse_interpolated_string`, which parses the entire string, to
`parse_interpolated_element`. This kind of seems more natural anyway and
avoids
having to try to recursively visit nested elements after the fact in
`parse_interpolated_string`. So viewing the diff with something like

```
git diff --color-moved --ignore-space-change --color-moved-ws=allow-indentation-change main
```

should make this more clear.

Test Plan
--

New tests
@BrewTestBot BrewTestBot mentioned this pull request Oct 23, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MichaReiser MichaReiser MichaReiser approved these changes

Assignees

No one assigned

Labels

bug Something isn't working formatter Related to the formatter

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Formatter can reuse nested quotes before 3.12

2 participants

@ntBre @MichaReiser