Polish some macro parsing code

Polish some macro parsing code



I'm working on some optimizations to `macro_parser.rs` and the surrounding code (specifically, changing the current BFS approach into a DFS), and these changes fell out during that effort. I think they make the code more accessible in addition to laying some foundations for my future work (in particular, the handling of ambiguities).

This is my first proper entrance into rustc code, please let me know if I'm holding something wrong :)

The commits are organized and individually reviewable.

r? @nnethercote

Preparatory changes for macro parsing BFS->DFS

This PR contains some miscellaneous commits I accumulated while working on the BFS->DFS change. Their goal is to simplify the code and clarifying existing behavior. It is a conceptual follow-up to #158577. High-level overview:

- Adds context about the current match arm to `Tracker` through the new `Tracker::prepare()`, so that the `WhichMatcher` parameter is available implicitly. In a later PR, this will be used to reference `MatcherLoc`s by index.

- Reformulates matching failures to work more like ambiguity errors wrt. `Tracker`; `Tracker::build_failure()` (which would build a failure consumed by `Tracker::after_arm()`) becomes `Tracker::failure()` which eagerly processes the error. This removes the need for `ParseResult::Failure` to store any data at all. This relies on the match arm context provided by `Tracker::prepare()`.

- There is a subtle edge case involving `token::Eof` and non-terminal parsing; `Parser::nonterminal_may_begin_with()` would sometimes return `true` for `token::Eof`, even though the non-terminal parse would never be attempted. `TtParser` did not check `bb_mps` when handling `token::Eof`, so non-terminal parses at EOF were being silently dropped. I changed `Parser::nonterminal_may_begin_with()` to always return `false` for `token::Eof`, making this behavior much more visible. I added a test case to make sure meta-variables return the same error uniformly when parsed against EOF.

Contains #158894, which should be merged soon (after which I'll rebase onto `main`).

Best reviewed commit-by-commit.

r? @nnethercote

Preparatory changes for macro parsing BFS->DFS

This PR contains some miscellaneous commits I accumulated while working on the BFS->DFS change. Their goal is to simplify the code and clarifying existing behavior. It is a conceptual follow-up to #158577. High-level overview:

- Adds context about the current match arm to `Tracker` through the new `Tracker::prepare()`, so that the `WhichMatcher` parameter is available implicitly. In a later PR, this will be used to reference `MatcherLoc`s by index.

- Reformulates matching failures to work more like ambiguity errors wrt. `Tracker`; `Tracker::build_failure()` (which would build a failure consumed by `Tracker::after_arm()`) becomes `Tracker::failure()` which eagerly processes the error. This removes the need for `ParseResult::Failure` to store any data at all. This relies on the match arm context provided by `Tracker::prepare()`.

- There is a subtle edge case involving `token::Eof` and non-terminal parsing; `Parser::nonterminal_may_begin_with()` would sometimes return `true` for `token::Eof`, even though the non-terminal parse would never be attempted. `TtParser` did not check `bb_mps` when handling `token::Eof`, so non-terminal parses at EOF were being silently dropped. I changed `Parser::nonterminal_may_begin_with()` to always return `false` for `token::Eof`, making this behavior much more visible. I added a test case to make sure meta-variables return the same error uniformly when parsed against EOF.

Contains #158894, which should be merged soon (after which I'll rebase onto `main`).

Best reviewed commit-by-commit.

r? @nnethercote

Preparatory changes for macro parsing BFS->DFS



This PR contains some miscellaneous commits I accumulated while working on the BFS->DFS change. Their goal is to simplify the code and clarifying existing behavior. It is a conceptual follow-up to #158577. High-level overview:

- Adds context about the current match arm to `Tracker` through the new `Tracker::prepare()`, so that the `WhichMatcher` parameter is available implicitly. In a later PR, this will be used to reference `MatcherLoc`s by index.

- Reformulates matching failures to work more like ambiguity errors wrt. `Tracker`; `Tracker::build_failure()` (which would build a failure consumed by `Tracker::after_arm()`) becomes `Tracker::failure()` which eagerly processes the error. This removes the need for `ParseResult::Failure` to store any data at all. This relies on the match arm context provided by `Tracker::prepare()`.

- There is a subtle edge case involving `token::Eof` and non-terminal parsing; `Parser::nonterminal_may_begin_with()` would sometimes return `true` for `token::Eof`, even though the non-terminal parse would never be attempted. `TtParser` did not check `bb_mps` when handling `token::Eof`, so non-terminal parses at EOF were being silently dropped. I changed `Parser::nonterminal_may_begin_with()` to always return `false` for `token::Eof`, making this behavior much more visible. I added a test case to make sure meta-variables return the same error uniformly when parsed against EOF.

Contains #158894, which should be merged soon (after which I'll rebase onto `main`).

Best reviewed commit-by-commit.

r? @nnethercote

Preparatory changes for macro parsing BFS->DFS



This PR contains some miscellaneous commits I accumulated while working on the BFS->DFS change. Their goal is to simplify the code and clarifying existing behavior. It is a conceptual follow-up to rust-lang/rust#158577. High-level overview:

- Adds context about the current match arm to `Tracker` through the new `Tracker::prepare()`, so that the `WhichMatcher` parameter is available implicitly. In a later PR, this will be used to reference `MatcherLoc`s by index.

- Reformulates matching failures to work more like ambiguity errors wrt. `Tracker`; `Tracker::build_failure()` (which would build a failure consumed by `Tracker::after_arm()`) becomes `Tracker::failure()` which eagerly processes the error. This removes the need for `ParseResult::Failure` to store any data at all. This relies on the match arm context provided by `Tracker::prepare()`.

- There is a subtle edge case involving `token::Eof` and non-terminal parsing; `Parser::nonterminal_may_begin_with()` would sometimes return `true` for `token::Eof`, even though the non-terminal parse would never be attempted. `TtParser` did not check `bb_mps` when handling `token::Eof`, so non-terminal parses at EOF were being silently dropped. I changed `Parser::nonterminal_may_begin_with()` to always return `false` for `token::Eof`, making this behavior much more visible. I added a test case to make sure meta-variables return the same error uniformly when parsed against EOF.

Contains rust-lang/rust#158894, which should be merged soon (after which I'll rebase onto `main`).

Best reviewed commit-by-commit.

r? @nnethercote