Skip to content

fix(yaml): preserve line numbers after header preprocessing - #2939

Merged
mikefarah merged 1 commit into
mikefarah:masterfrom
imharjot:fix-yaml-lines
Oct 26, 2026
Merged

mikefarah merged 1 commit into
mikefarah:masterfrom
imharjot:fix-yaml-lines

Conversation

@imharjot

Copy link
Copy Markdown
Contributor

Prepared by the Codex AI agent acting on behalf of imharjot, not written by the user personally.

Fixes #1956.

Header preprocessing removes comments, blank lines and separators before YAML decoding, so line reports positions relative to the remaining input. The reported .foo.bar | line example returns 2 instead of 4.

Keep those consumed lines as blank parser input while retaining the original header for output. Also keep the newline when reinserting content after an inline ---, so an inline comment cannot swallow the following line.

Regressions cover LF/CRLF headers, keys and values, later documents, decoder reuse, inline separators, and comment/content preservation. They fail with the original decoder. The built CLI reproduces the before/after results.

Validation on macOS arm64: the focused line/YAML tests, alternate GOCCY=true line tests, formatting, spelling, gosec, lint, build, Go-install compatibility and acceptance tests pass.

The full unit suite has one pre-existing failure: TestJSONScenarios expects json: string of object unexpected end of JSON input, but gets invalid character 'b' after object key:value pair. Unchanged master and this patch fail identically on Go 1.26.4 and 1.27.1 (the development-container pin). No assertions were changed. make local build exits successfully because scripts/test.sh continues after the failing go test; I also ran go test ./... directly to check this.

@mikefarah mikefarah added the popular Affects multiple users label Oct 26, 2026
@mikefarah

Copy link
Copy Markdown
Owner

@mikefarah Triage: this looks like a popular PR. 5 distinct outside users (all on #1956, raised by martincostello): wrong line numbers reported for YAML docs with headers/directives; breaks error locations in tooling.

@mikefarah
mikefarah merged commit f488e74 into mikefarah:master Oct 26, 2026
4 checks passed
gabrielcosi pushed a commit to gabrielcosi/home-ops that referenced this pull request Nov 6, 2026
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [mikefarah/yq](https://mikefarah.gitbook.io/yq/) ([source](https://github.com/mikefarah/yq)) | patch | `4.54.1` → `4.54.2` |

---

### Release Notes

<details>
<summary>mikefarah/yq</summary>

### [`v4.54.2`](https://github.com/mikefarah/yq/releases/tag/v4.54.2)

[Compare Source](mikefarah/yq@v4.54.1...v4.54.2)

- Fixed command injection risk in shell output with `-o=shell -r`: unsafe values are now quoted and a warning is logged. Use `--security-force-never-escape-shell-output` to restore the previous unsafe behaviour (GHSA-h2cv-65fq-jgrp)
  - Fixed YAML structure being changed when assigning head comments ([#&#8203;2847](mikefarah/yq#2847)) Thanks [@&#8203;rootsec1](https://github.com/rootsec1)
  - Fixed `ireduce` to run per pipeline input, matching jq's `reduce` behaviour ([#&#8203;2869](mikefarah/yq#2869)) Thanks [@&#8203;00200200](https://github.com/00200200)
  - Fixed `map_values` to remove entries when the expression returns nothing ([#&#8203;2862](mikefarah/yq#2862)) Thanks [@&#8203;yuefdev](https://github.com/yuefdev)
  - Fixed preservation of comments on TOML inline table values ([#&#8203;2879](mikefarah/yq#2879)) Thanks [@&#8203;iamhuman-cheolheelee](https://github.com/iamhuman-cheolheelee)
  - Fixed INI keys containing colons ([#&#8203;2886](mikefarah/yq#2886))
  - Fixed YAML line numbers after header preprocessing ([#&#8203;2939](mikefarah/yq#2939)) Thanks [@&#8203;imharjot](https://github.com/imharjot)
  - Fixed an extra leading comment blank line with CRLF input ([#&#8203;2856](mikefarah/yq#2856)) Thanks [@&#8203;cestercian](https://github.com/cestercian)
  - Cleaned up failed in-place edits ([#&#8203;2920](mikefarah/yq#2920)) Thanks [@&#8203;Boulea7](https://github.com/Boulea7)
  - Bumped dependencies

</details>

---

### Configuration

📅 **Schedule**: (in timezone Europe/Berlin)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xOTguMyIsInVwZGF0ZWRJblZlciI6IjQ0LjE5OC4zIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL3BhdGNoIl19-->

Reviewed-on: https://git.xcd.dev/gabrielcosi/home-ops/pulls/1604
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

popular Affects multiple users

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Line numbers reported incorrectly when YAML document starts with trivia

2 participants