Repository navigation
codegen: CRLF and lone CR line endings (#230, line-ending part) - #252
Merged
Merged
Conversation
Signed-off-by: luisleo526 <[email protected]>
Normalize CRLF and lone CR code/comment line endings in Lexer initialization after checking the original source size. Construct that lexer at the front of _parse_bounded and pass lexer.source to pragma extraction and Parser, so transpile, transpile_full, the request inventory API, and JSON/Pyodide glue use the same text and locations. Keeping the rule in Lexer also covers direct token users and expression reparsing. Preserve quoted string contents, including raw CR characters, escaped delimiters and triple-quoted strings. LF source passes through unchanged. Add an Unreleased changelog entry and update the matching project memory. Regression tests are in the preceding tests-only commit. No tests or builds were run on this Mac, as instructed; validation here was static Python syntax parsing and git diff checks only. Signed-off-by: luisleo526 <[email protected]>
A CRLF pair is a line terminator wherever it appears, including inside a multiline string, a wrapped one-quote string and right after a backslash, so a script saved with CRLF now yields the same string values, C++ and diagnostics as the LF form. A lone CR inside a string stays data (the committed regression tests pin it); lone-CR-only files with a multiline or wrapped string are a documented limit. Also build the pragma pre-pass lexer inside its try, so a whole-source size error keeps the recovery it had before the size check moved into the lexer. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…#230) A global replace of CRLF turned "\r\r\n" (a lone CR followed by a CRLF, the artifact of a file converted twice) into CR+LF, which the scanner then read as one break where the LF reading has two: a blank line was lost, later diagnostic line numbers were one short and a comment line merged with the next. The CRLF pair is now folded only inside the scanner's string states (and right after a backslash), and outside strings the existing branch takes CRLF and a lone CR as one break each, in order. A CRLF inside a one-quote string applies the same unwrapped-line recovery as the LF form, and that recovery also treats a following CRLF blank line as blank. Tests compare tokens, C++ and diagnostics with the LF form of the same script for "\r\r\n" after code, after a comment and inside both string kinds. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Brings the branch up to date with codegen main (CHANGELOG backfill, #248). The merge is clean: CHANGELOG.md auto-merged and keeps both main's entries and the Unreleased note for CRLF and CR line endings (#230). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The two double-CRLF translate tests used scripts without //@Version=6, so the LF form was refused for the missing directive before it could be compared. They now start with the directive like the other tests in the file, and the line-three diagnostics test does too, so its syntax error is the one compared (not the missing-directive refusal). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
# Conflicts: # CHANGELOG.md
This was referenced Oct 26, 2026
luisleo526
added a commit
that referenced
this pull request
Oct 26, 2026
## What Replaces the three-bullet `## Unreleased` block of `CHANGELOG.md` with the full planned-1.8.0 entry and its migration notes. `CHANGELOG.md` is the only file changed (+54/-10). Documentation only. ## Why The release workflow creates the GitHub release with a fixed install line and no migration notes; the notes users read are this file. They must be on main before the release dispatch. The heading stays `## Unreleased` until the `v1.8.0` tag exists, as the file's own release note policy says; a separate docs-only commit dates it after the tag. ## Content - Compatibility and migration: exact pairing with engine `v1.8.0`; regenerate strategy C++ and relink; the new `color` input type with its unsigned packed `0xAARRGGBB` default that hosts must preserve; the `strategy.account_currency` emitted-C++ change (intended to change only for scripts that read it; not a byte comparison against 1.7.0); CRLF and lone CR line endings accepted. - The four pull requests merged after `v1.7.0`: #251 (#231), #250 (#227), #252 (#230), #248 (CHANGELOG backfill, no behaviour change). - One sentence about pineforge-hpo accepting color inputs from its 1.8.0 release; true of hpo main today, re-checked against hpo's 1.8.0 entry before the release. ## Checks - Read as a committed diff by an independent reviewer: GO. The four commits since `v1.7.0` are the four bullets. - The `color.red` value `4294063685` (`0xFFF23645`) was recomputed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Supersedes #256 (same change; the commit message no longer carries an internal tracking id).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #230 (CRLF / CR part). The lexer folds
\r\nto\n(also inside strings) and treats a lone\ras a line ending in code and comments, with the same output and line numbers as the LF form. A lone\rinside a string literal stays data. The normalized source feeds pragma extraction and the parser too; directLexercallers share the rule (docs in AGENTS.md / CLAUDE.md contract table + CHANGELOG).Tests:
tests/test_line_endings.py(expectations computed from the LF form of the same script) + glue/version-directive updates. Reviewed (REV-CX GO, four rounds). Part 1 of #230 (:=on a UDT field in a single-line function body) is a separate PR.🤖 Generated with Claude Code