yq-keep

Apply a yq expression to YAML files in place and change only the lines the edit touches. Blank lines, comment spacing, indentation style and flow-sequence spacing elsewhere in the file stay byte-for-byte as they were.

$ cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- deployment.yaml   # main
- service.yaml

images:
    - name: app
      newTag: "1.2.3"

$ yq-keep '.resources += ["rg.yaml"] | .images[0].newTag = "1.3.0"' kustomization.yaml
$ git diff -U1
@@ -6,2 +6,3 @@ resources:
 - service.yaml
+- rg.yaml

@@ -9,2 +10,2 @@ images:
     - name: app
-      newTag: "1.2.3"
+      newTag: "1.3.0"

The same edit with yq -i changes 11 lines: it also removes both blank lines, re-indents the sequences under resources and images, and collapses the spaces before # main.

Why

yq parses YAML into a node tree and writes the tree back out. The parser does not keep blank lines or the original indentation, so yq -i reformats the whole file. Users who edit hand-maintained files (Kubernetes manifests, Helm values, CI configs, Compose files) get large diffs from a one-line change. The requests have been open for years:

• mikefarah/yq#515 "yq write strips completely blank lines from the output": 156 reactions, 41 comments, open since August 2020.
• mikefarah/yq#465 "Preserve formatting with in place writing": 116 reactions, open since June 2020.
• mikefarah/yq#1359 "Preserve indent option for write in place": 37 reactions.
• mikefarah/yq#462 "Preserve original indentation level for list items": 26 reactions, 18 comments.

The maintainer traced it to the YAML library (go-yaml/yaml#627); a fix in the library is in progress but not released. The most-upvoted workaround in #515 (61 reactions) pipes yq's output through diff | patch. Later comments in the same thread report that it drops comments, merges list items or leaves stray blank lines depending on where the edit lands, and the variants posted since are shell functions without tests. Format-preserving editors exist (yamlpath, built on ruamel.yaml), but they use their own query syntax, so existing yq expressions and scripts have to be rewritten.

yq-keep keeps yq as the engine: any expression that works with yq -i can be passed unchanged.

Installation

Requires Python 3.10+ and mikefarah/yq v4 on PATH (or set $YQ / --yq).

pipx install git+https://github.com/Jwrede/yq-keep
# or run without installing
uvx --from git+https://github.com/Jwrede/yq-keep yq-keep --help

Usage

yq-keep [--diff] [--check] [--yq PATH] EXPRESSION FILE [FILE ...]

• Without flags, each file is rewritten in place, like yq -i EXPRESSION FILE.
• --diff prints a unified diff and writes nothing.
• --check writes nothing and exits 1 if any file would change (for CI).
• Exit code 2 means yq failed or a file was refused (see below); other files are still processed.

Expressions can read environment variables the usual yq way:

TAG=1.4.0 yq-keep '.image.tag = strenv(TAG)' values.yaml

How it works

1. yq '.' FILE gives yq's rendering of the unchanged file (base).
2. yq EXPRESSION FILE gives the rendering after the edit (new). Both share yq's formatting, so the line diff between them contains only the edit.
3. The lines of base are aligned with the original file, ignoring blank lines and whitespace differences. Each hunk of the diff is applied at the aligned position in the original.
4. Inserted lines are re-indented to match their nearest sibling in the original (so a file that writes sequences as - item directly under the key, or uses four-space indentation, keeps that style).
5. The result is parsed again with yq and compared to new as data (JSON). If they differ, the file is left unchanged and yq-keep exits 2. A format-preserving result is never written unless it holds exactly the data yq -i would have written.

Line endings (LF or CRLF) and a missing final newline are kept.

When a file is refused, the message says why:

• yq's own round trip (yq '.') changes the data of this file, so yq -i would too: yq itself mis-renders something in the file (seen with some folded >- block scalars). Plain yq -i would silently change the data here; yq-keep writes nothing.
• the format-preserving result does not parse to the same data as yq's output: the line mapping failed for this edit. Use plain yq -i for it.

Testing on real files

Besides the unit tests, yq-keep was run against 361 YAML files from a development machine (Kubernetes manifests and Helm templates, GitHub and GitLab CI workflows, Compose files, pre-commit configs) with five edits each: add a top-level key, change the first string, delete the first key, append to the first list of strings, and sort_keys(..).

• 1,621 of 1,636 edits were written. Of the 15 refusals, 11 were files where yq's own round trip already changes the data, 2 were a block scalar on the last line of a file without a final newline, and 2 were sort_keys(..) on GitLab CI files.
• For the 1,088 edits other than sort_keys that changed a file, yq-keep changed 2,819 lines in total; yq -i changed 26,634 lines for the same edits.

Limitations

• Lines the edit touches are taken from yq's output, so a changed line gets yq's spacing (for example one space before its trailing comment).
• Edits that reorder or rewrite most of a file (sort_keys(..), renaming every key) produce correct data, but the blank lines stay at their old positions and can end up in odd places.
• Deleting a block leaves the blank lines that surrounded it.
• Comments that yq itself moves or drops (yq has known issues with some comment positions) are handled by the same alignment; where yq loses a comment in a changed region, yq-keep loses it too.
• Appending after a block scalar on the last line of a file without a final newline changes that scalar's value; the check catches this and the file is refused.
• Only YAML input is supported. Front matter and other yq input formats are not handled.
• yq is run twice per file plus twice for the check, so it is slower than yq -i on very large files.

Development

python -m venv .venv && .venv/bin/pip install -e ".[dev]"
.venv/bin/pytest -v

The tests run the real yq binary; CI installs yq v4.

License

MIT