Open
Conversation
haampie
force-pushed
the
hs/fix/parser-speed-2
branch
2 times, most recently
from
f63673b to
5f4245e
Compare
This comment was marked as outdated.
This comment was marked as outdated.
haampie
force-pushed
the
hs/fix/parser-speed-2
branch
2 times, most recently
from
6bfbe67 to
f598fc8
Compare
This was referenced Jan 13, 2026
haampie
force-pushed
the
hs/fix/parser-speed-2
branch
from
7234377 to
c7c17d4
Compare
haampie
force-pushed
the
hs/fix/parser-speed-2
branch
2 times, most recently
from
8aa564b to
153c321
Compare
haampie
force-pushed
the
hs/fix/parser-speed-2
branch
2 times, most recently
from
75bf4c7 to
6f21610
Compare
Baseline on upstream/develop: 1782.62 μs Signed-off-by: Harmen Stoppels <[email protected]>
…f parser total time: 1716.36 μs (prev: 1782.62 μs) Signed-off-by: Harmen Stoppels <[email protected]>
…(no callers yet) total time: 1704.15 μs (prev: 1716.36 μs) Signed-off-by: Harmen Stoppels <[email protected]>
Signed-off-by: Harmen Stoppels <[email protected]>
- Add named subgroups to VERSION, BOOL_VARIANT, KEY_VALUE_PAIR, PROPAGATED_BOOL_VARIANT, PROPAGATED_KEY_VALUE_PAIR, VERSION_HASH_PAIR, GIT_VERSION token patterns - Use token.subvalues in SpecNodeParser and EdgeAttributeParser instead of slicing token.value - Use VersionList._from_version_list_string() for VERSION tokens - Fix tokenize.py Token.__eq__ to treat None subvalues as wildcard total time: 1671.55 μs (prev: 1683.70 μs) Signed-off-by: Harmen Stoppels <[email protected]>
- Merge VERSION_HASH_PAIR, GIT_VERSION, VERSION into single VERSION token (git_version group present iff it is a git version, else version_list) - Merge PROPAGATED_BOOL_VARIANT into BOOL_VARIANT (bv_prefix length distinguishes) - Merge PROPAGATED_KEY_VALUE_PAIR into KEY_VALUE_PAIR (kv_sep contains == for propagated) - Merge START_EDGE_PROPERTIES into DEPENDENCY (edge_bracket group present iff ^[/%%[/%[) - tokenize.py: only store non-None subvalues in token.subvalues - Update spec_syntax.py tests to use consolidated token names total time: 1517.40 μs (prev: 1671.55 μs) Signed-off-by: Harmen Stoppels <[email protected]>
…ll arch shift Signed-off-by: Harmen Stoppels <[email protected]>
Signed-off-by: Harmen Stoppels <[email protected]>
total time: 1510.42 μs (prev: 1517.40 μs) Signed-off-by: Harmen Stoppels <[email protected]>
…ect dispatch total time: 1289.36 μs (prev: 1510.42 μs) Signed-off-by: Harmen Stoppels <[email protected]>
…, no generator Replace the generator-based token stream in TokenContext with a direct scanner driven by SPEC_TOKENIZER.regex.scanner(). Advance skips WS and UNEXPECTED checks inline; current_token/next_token now hold re.Match objects. All dispatch sites updated from Token.kind/subvalues to Match.lastgroup/group(). _parse_edge_attrs returns (attributes, substitute) tuple to avoid push_front of synthetic Token. total time: 966.24 μs (prev: 1289.36 μs) Signed-off-by: Harmen Stoppels <[email protected]>
Build FAST_SPEC_REGEX with a leading \s* so whitespace is consumed at the C level before every token match. Drop TokenContext entirely; SpecParser now holds curr/next re.Match slots and advances inline. tokens() returns (kind, value, subgroups) tuples; SpecTokens becomes a plain class of regex strings; SpecTokenizationError re-scans via FAST_SPEC_REGEX. Cache the Spec class in a module-level _Spec variable to avoid repeated import lookups in _parse_node. total time: 875.54 μs (prev: 966.24 μs) Signed-off-by: Harmen Stoppels <[email protected]>
Signed-off-by: Harmen Stoppels <[email protected]>
Pure reorder: move SpecTokenizationError, parse(), parse_one_or_raise(), toolchain helpers, SpecParsingError, strip_quotes_and_unescape(), and quote_if_needed() before SpecTokens/FAST_SPEC_REGEX. Also fix conf.py to import GIT_VERSION_PATTERN, NAME, VERSION, VERSION_LIST from spec_parser (needed for Sphinx docs). Restore tokenize.py to reference state. No functional changes. Brings tree to exact parity with ea96bb6885884b19d18e4517b9bd76d7eef75d13. total time: 863.45 μs (prev: 875.54 μs) Signed-off-by: Harmen Stoppels <[email protected]>
Signed-off-by: Harmen Stoppels <[email protected]>
Signed-off-by: Harmen Stoppels <[email protected]>
haampie
force-pushed
the
hs/fix/parser-speed-2
branch
from
6f21610 to
75941ca
Compare
Replace the custom fast-path with VersionList(version_list_string), which routes through the existing from_string() path. The -12 µs saving was not worth the added surface area. total time: 887.56 µs (prev: 863.45 µs) Signed-off-by: Harmen Stoppels <[email protected]>
haampie
force-pushed
the
hs/fix/parser-speed-2
branch
from
c1dcef7 to
ab2b8a4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tokenwrapper objects with direct usage ofre.Matchobjects; removedTokenContextobject creation.accept/expectlogic; removedSpecNodeParser,EdgeAttributeParser, andFileParserclasses, flattening the recursive descent into a singleSpecParserclass.WStoken; whitespace is now consumed implicitly by the regex engine (C-level) via\s*prefix.WS, similar tokens (VERSION/GIT,BOOL/PROPAGATED,DEPENDENCY/START_EDGE_PROPERTIES) are consolidated to reduce branching in parser loop.bv_prefix,kv_value) are used extract data during matching, avoiding double parsing (notably versions). Also avoid further allocations that exist on develop where named capture groups names had a prefix removed for convenience, resulting in another dictionary per token when named capture groups were used.benchmark: https://gist.github.com/haampie/fec012851209b7da62ca78b62d31ce87
breakdown