Skip to content

Parser error on await ... #4672

@MeGaGiGaGon

Description

@MeGaGiGaGon

The code await ... crashes the parser with error: cannot format <string>: '...' locally / misparses into '...' online (not sure why the difference here, I think directly running __init__.py has more dev checks enabled?).

This happens because tokenize.tokenize has this structure (taking some creative liberties with the yields):

for token in token_iterator:
    if token.type == TokenType.identifier and token_str in ("async", "await"):
        yield current_token
        yield next_token
        continue
    if token.type == TokenType.op and token_str == "...":
        # convert the `...` to three `.` tokens
        yield new_token # x3
        continue

This means that on an await token, the next token bypasses the ... check, so the parser gets an op ... instead of the expected 3x . ops.

I'm not sure how to best fix this without making a big mess of the code, @tusharsadhwani please help.

Metadata

Metadata

Assignees

No one assigned

    Labels

    T: bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions