JSON store: preserve large integers that fit into int64 by s3onghyun · Pull Request #2222 · getsops/sops

s3onghyun · 2026-06-18T12:22:32Z

Problem

The JSON store decodes with json.Decoder but never calls UseNumber(), so every JSON number is decoded into a Go float64. float64 holds only ~15–17 significant decimal digits, so any integer above 2^53 (~9e15) is silently corrupted on re-emit:

input :  {"id": 1234567890123456789}
output:  {"id": 1234567890123456800}

This hits very common secret payloads — Discord/Twitter snowflake IDs, 64-bit identifiers, unix-nanosecond timestamps, large account/order numbers. In a secrets-management tool, a value silently changing across an encrypt/decrypt round-trip is serious.

Fix

Call dec.UseNumber() on the decoder in treeBranchFromJSON. Numbers then arrive as json.Number (a string type) and are re-emitted verbatim. The recursive decode helpers share the one decoder, so a single call covers the whole tree. stores.ValToString already stringifies json.Number via fmt.Stringer for encryption/MAC, so no core-path changes are needed.

Testing

Added TestLargeIntegerRoundtrip (stores/json/store_test.go): a 1234567890123456789 value round-trips exactly. Fails before (mangled to ...800), passes after.
Updated existing tests that hard-coded the old lossy float64 representation to the correct lossless json.Number.
go test ./stores/... . green.

felixfontein · 2026-06-18T20:11:31Z

I don't think this PR works, as you can use different stores for reading and writing, and no other store than JSON can handle json.Number values. Also can these actually be encrypted?

felixfontein · 2026-06-18T20:21:23Z

I guess the answer is no, as several JSON related tests failed.

s3onghyun · 2026-06-19T00:28:39Z

Good catch, thanks @felixfontein — you're right on both counts. json.Number leaks into the store-agnostic tree (so reading with JSON and writing with another store breaks) and the AES cipher can't encrypt it (it only handles string/int/float64/bool/time/comment), which is what the failing JSON tests showed.

I've reworked it: decode with UseNumber but immediately normalize each number to a concrete type the whole codebase already handles — int for integers and float64 otherwise. This:

keeps the tree free of json.Number, so cross-store read/write works;
is encryptable (int/float64 are handled by the cipher);
matches how the YAML store already decodes integers (as int), so JSON and YAML are now consistent;
still fixes the original precision loss (e.g. 1234567890123456789 no longer becomes ...800).

I added an encrypt → decrypt round-trip assertion to the regression test to prove the decoded value is actually encryptable. go test ./stores/... ./aes/... . is green. PTAL.

felixfontein · 2026-06-19T05:47:28Z

+	if f, err := n.Float64(); err == nil {
+		return f
+	}
+	return t


I don't think returning the json.Number object as a fallback is a good idea. Instead an error should be returned, namely the one from convering it to a float64.

s3onghyun · 2026-06-19T05:50:36Z

Good point, thanks — you're right that falling back to the json.Number would re-introduce exactly the leak this is meant to avoid. Updated normalizeJSONNumber to return (interface{}, error) and propagate the json.Number.Float64() error when a value can be represented as neither int nor float64; both call sites now bubble it up. (In practice the decoder only ever yields syntactically-valid numbers so this path is unreachable, but returning the error is the correct, leak-free behavior.) go test ./stores/... ./aes/... . stays green.

s3onghyun · 2026-06-19T05:54:34Z

Done. A couple of things I verified while at it, to be explicit about the boundaries:

Range: integers within the int64 range now stay exact (covers all 64-bit signed IDs, snowflakes, etc.). Values above int64 fall back to float64 — bounded by the cipher's supported numeric types (int and float64). That's still strictly better than before, where every integer became a float64 and lost precision above 2^53. I documented this on normalizeJSONNumber and added TestIntegerBoundaries covering negative, MinInt64, MaxInt64, and zero round-tripping exactly as int.
Cross-store: confirmed the decoded int/float64 values emit cleanly through the YAML store (no json.Number reaches a non-JSON store), e.g. 1234567890123456789 (int) → YAML 1234567890123456789.

go test ./stores/... ./aes/... . stays green. Open to a different approach for the >int64 range if you'd prefer (the cipher would need to grow a type to do better than float64 there).

s3onghyun · 2026-06-19T06:47:07Z

Rebased onto the latest main and squashed the review iterations into a single commit. Tests stay green (go test ./stores/... ./aes/... .). Ready for another look whenever you have time — thanks for the thorough review, it made the change considerably more robust.

felixfontein

Thanks, this is definitely an improvement.

felixfontein · 2026-06-20T13:39:12Z

+	cipher := aes.NewCipher()
+	key := []byte("0123456789abcdef0123456789abcdef")
+	enc, err := cipher.Encrypt(value, key, "")
+	assert.Nil(t, err)
+	assert.Contains(t, enc, "type:int")
+	dec, err := cipher.Decrypt(enc, key, "")
+	assert.Nil(t, err)
+	assert.Equal(t, value, dec)


I would remove this test, since it doesn't really test something related to JSON.

Suggested change

cipher := aes.NewCipher()

key := []byte("0123456789abcdef0123456789abcdef")

enc, err := cipher.Encrypt(value, key, "")

assert.Nil(t, err)

assert.Contains(t, enc, "type:int")

dec, err := cipher.Decrypt(enc, key, "")

assert.Nil(t, err)

assert.Equal(t, value, dec)

felixfontein · 2026-06-20T13:40:29Z

+// Decoding integers as int (rather than the encoding/json default float64)
+// preserves large-integer precision and matches how the YAML store decodes
+// them. The AES cipher only supports int and float64 for numbers, so integers
+// within the int64 range are kept exact while values outside it fall back to
+// float64 — still an improvement over the previous behavior, where every
+// integer became a float64 and lost precision above 2^53.


I think this text would be fine for a commit message, but it doesn't really belong in the comment.

Suggested change

// Decoding integers as int (rather than the encoding/json default float64)

// preserves large-integer precision and matches how the YAML store decodes

// them. The AES cipher only supports int and float64 for numbers, so integers

// within the int64 range are kept exact while values outside it fall back to

// float64 — still an improvement over the previous behavior, where every

// integer became a float64 and lost precision above 2^53.

The JSON store decoded numbers with encoding/json's default behavior, which turns every number into a float64. Integers above 2^53 (snowflake IDs, 64-bit identifiers, unix-nanosecond timestamps, large account numbers) were silently corrupted across an encrypt/decrypt round-trip, e.g. 1234567890123456789 became 1234567890123456800 — silent data corruption in a secrets tool. Decode with UseNumber and normalize each number to a concrete type the whole codebase (every store and the AES cipher) handles: an int for integers within the int64 range, a float64 otherwise. This keeps json.Number out of the store-agnostic tree (so reading with JSON and writing with another store works), is encryptable (the cipher supports int/float64), matches how the YAML store decodes integers, and preserves precision for all 64-bit IDs. Values outside the int64 range fall back to float64 — still better than the previous all-float64 behavior. If a number can be represented as neither, the json.Number.Float64 error is propagated rather than leaking a json.Number. Adds a large-integer encrypt/decrypt round-trip test and int64 boundary tests (negative, MinInt64, MaxInt64, zero). Signed-off-by: Seonghyun Hong <[email protected]>

s3onghyun · 2026-06-20T13:55:02Z

Thanks, @felixfontein — both applied:

Removed the cipher encrypt/decrypt assertions from the test; it now only checks the JSON-store concern (large integers round-trip and decode to a concrete int, not a json.Number).
Trimmed the doc comment on normalizeJSONNumber to just what it does; the rationale (precision above 2^53, parity with the YAML store) now lives in the commit message instead.

go test ./stores/json/ passes.

felixfontein

Thanks a lot!

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [getsops/sops](https://github.com/getsops/sops) | patch | `v3.13.1` → `v3.13.2` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>getsops/sops (getsops/sops)</summary> ### [`v3.13.2`](https://github.com/getsops/sops/releases/tag/v3.13.2) [Compare Source](getsops/sops@v3.13.1...v3.13.2) #### Installation To install `sops`, download one of the pre-built binaries provided for your platform from the artifacts attached to this release. For instance, if you are using Linux on an AMD64 architecture: ```shell # Download the binary curl -LO https://github.com/getsops/sops/releases/download/v3.13.2/sops-v3.13.2.linux.amd64 # Move the binary in to your PATH mv sops-v3.13.2.linux.amd64 /usr/local/bin/sops # Make the binary executable chmod +x /usr/local/bin/sops ``` ##### Verify checksums file signature The checksums file provided within the artifacts attached to this release is signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of this file, run the following commands: ```shell # Download the checksums file, certificate and signature curl -LO https://github.com/getsops/sops/releases/download/v3.13.2/sops-v3.13.2.checksums.txt curl -LO https://github.com/getsops/sops/releases/download/v3.13.2/sops-v3.13.2.checksums.sigstore.json # Verify the checksums file cosign verify-blob sops-v3.13.2.checksums.txt \ --bundle sops-v3.13.2.checksums.sigstore.json \ --certificate-identity-regexp=https://github.com/getsops \ --certificate-oidc-issuer=https://token.actions.githubusercontent.com ``` ##### Verify binary integrity To verify the integrity of the downloaded binary, you can utilize the checksums file after having validated its signature: ```shell # Verify the binary using the checksums file sha256sum -c sops-v3.13.2.checksums.txt --ignore-missing ``` ##### Verify artifact provenance The [SLSA provenance](https://slsa.dev/provenance/v0.2) of the binaries, packages, and SBOMs can be found within the artifacts associated with this release. It is presented through an [in-toto](https://in-toto.io/) link metadata file named `sops-v3.13.2.intoto.jsonl`. To verify the provenance of an artifact, you can utilize the [`slsa-verifier`](https://github.com/slsa-framework/slsa-verifier#artifacts) tool: ```shell # Download the metadata file curl -LO https://github.com/getsops/sops/releases/download/v3.13.2/sops-v3.13.2.intoto.jsonl # Verify the provenance of the artifact slsa-verifier verify-artifact <artifact> \ --provenance-path sops-v3.13.2.intoto.jsonl \ --source-uri github.com/getsops/sops \ --source-tag v3.13.2 ``` #### Container Images The `sops` binaries are also available as container images, based on Debian (slim) and Alpine Linux. The Debian-based container images include any dependencies which may be required to make use of certain key services, such as GnuPG, AWS KMS, Azure Key Vault, and Google Cloud KMS. The Alpine-based container images are smaller in size, but do not include these dependencies. These container images are available for the following architectures: `linux/amd64` and `linux/arm64`. ##### GitHub Container Registry - `ghcr.io/getsops/sops:v3.13.2` - `ghcr.io/getsops/sops:v3.13.2-alpine` ##### Quay.io - `quay.io/getsops/sops:v3.13.2` - `quay.io/getsops/sops:v3.13.2-alpine` ##### Verify container image signature The container images are signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of an image, run the following command: ```shell cosign verify ghcr.io/getsops/sops:v3.13.2 \ --certificate-identity-regexp=https://github.com/getsops \ --certificate-oidc-issuer=https://token.actions.githubusercontent.com \ -o text ``` ##### Verify container image provenance The container images include [SLSA provenance](https://slsa.dev/provenance/v0.2) attestations. For more information around the verification of this, please refer to the [`slsa-verifier` documentation](https://github.com/slsa-framework/slsa-verifier#containers). #### Software Bill of Materials The Software Bill of Materials (SBOM) for each binary is accessible within the artifacts enclosed with this release. It is presented as an [SPDX](https://spdx.dev/) JSON file, formatted as `<binary>.spdx.sbom.json`. #### What's Changed - build(deps): Bump the go group with 3 updates by [@dependabot](https://github.com/dependabot)\[bot] in [#2185](getsops/sops#2185) - build(deps): Bump github/codeql-action from 4.35.4 to 4.35.5 in the ci group by [@dependabot](https://github.com/dependabot)\[bot] in [#2184](getsops/sops#2184) - build(deps): Bump the go group with 11 updates by [@dependabot](https://github.com/dependabot)\[bot] in [#2193](getsops/sops#2193) - build(deps): Bump the ci group with 4 updates by [@dependabot](https://github.com/dependabot)\[bot] in [#2192](getsops/sops#2192) - build(deps): Bump serde\_json from 1.0.149 to 1.0.150 in /functional-tests in the rust group by [@dependabot](https://github.com/dependabot)\[bot] in [#2191](getsops/sops#2191) - build(deps): Bump the go group with 11 updates by [@dependabot](https://github.com/dependabot)\[bot] in [#2197](getsops/sops#2197) - build(deps): Bump docker/setup-qemu-action from 4.0.0 to 4.1.0 in the ci group by [@dependabot](https://github.com/dependabot)\[bot] in [#2196](getsops/sops#2196) - test: unset all age env vars in make test target by [@arpitjain099](https://github.com/arpitjain099) in [#2208](getsops/sops#2208) - build(deps): Bump the go group with 10 updates by [@dependabot](https://github.com/dependabot)\[bot] in [#2212](getsops/sops#2212) - build(deps): Bump the ci group with 2 updates by [@dependabot](https://github.com/dependabot)\[bot] in [#2211](getsops/sops#2211) - build(deps): Bump the go group with 15 updates by [@dependabot](https://github.com/dependabot)\[bot] in [#2218](getsops/sops#2218) - build(deps): Bump alpine from 3.23 to 3.24 in /.release in the docker group by [@dependabot](https://github.com/dependabot)\[bot] in [#2217](getsops/sops#2217) - fix: handle pointers when serializing context by [@tlercher](https://github.com/tlercher) in [#2219](getsops/sops#2219) - docs: fix typo in exec-file --filename help text by [@s3onghyun](https://github.com/s3onghyun) in [#2221](getsops/sops#2221) - JSON store: preserve large integers that fit into int64 by [@s3onghyun](https://github.com/s3onghyun) in [#2222](getsops/sops#2222) - Fix panic when expecting an encrypted string, but a non-string is encountered by [@felixfontein](https://github.com/felixfontein) in [#2227](getsops/sops#2227) - Fix INI store no longer double-encoding newlines by [@felixfontein](https://github.com/felixfontein) in [#2189](getsops/sops#2189) - exec-file/exec-env: reset supplementary groups when changing user by [@felixfontein](https://github.com/felixfontein) in [#2194](getsops/sops#2194) - build(deps): Bump the go group with 6 updates by [@dependabot](https://github.com/dependabot)\[bot] in [#2229](getsops/sops#2229) - build(deps): Bump actions/checkout from 6.0.3 to 7.0.0 in the ci group by [@dependabot](https://github.com/dependabot)\[bot] in [#2228](getsops/sops#2228) - Fix issue when changing user in exec subcommands by [@sabre1041](https://github.com/sabre1041) in [#2230](getsops/sops#2230) - Shorten .md lines by [@felixfontein](https://github.com/felixfontein) in [#2206](getsops/sops#2206) - Update all Go dependencies with `go get -t -u ./...` by [@felixfontein](https://github.com/felixfontein) in [#2231](getsops/sops#2231) - build(deps): Bump github.com/opencontainers/runc from 1.2.8 to 1.3.6 by [@dependabot](https://github.com/dependabot)\[bot] in [#2233](getsops/sops#2233) - build(deps): Bump the ci group with 2 updates by [@dependabot](https://github.com/dependabot)\[bot] in [#2236](getsops/sops#2236) - Release 3.13.2 by [@felixfontein](https://github.com/felixfontein) in [#2232](getsops/sops#2232) #### New Contributors - [@arpitjain099](https://github.com/arpitjain099) made their first contribution in [#2208](getsops/sops#2208) - [@tlercher](https://github.com/tlercher) made their first contribution in [#2219](getsops/sops#2219) - [@s3onghyun](https://github.com/s3onghyun) made their first contribution in [#2221](getsops/sops#2221) **Full Changelog**: <getsops/sops@v3.13.1...v3.13.2> </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever MR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).

felixfontein reviewed Jun 19, 2026

View reviewed changes

s3onghyun force-pushed the fix-json-large-int-precision branch from 305be55 to 5d6bd13 Compare June 19, 2026 06:46

felixfontein reviewed Jun 20, 2026

View reviewed changes

s3onghyun force-pushed the fix-json-large-int-precision branch from 5d6bd13 to 014b5c7 Compare June 20, 2026 13:54

felixfontein changed the title ~~fix(json): preserve large integers losslessly via UseNumber~~ JSON store: preserve large integers losslessly via UseNumber Jun 20, 2026

felixfontein changed the title ~~JSON store: preserve large integers losslessly via UseNumber~~ JSON store: preserve large integers that fit into int64 Jun 20, 2026

felixfontein approved these changes Jun 20, 2026

View reviewed changes

felixfontein merged commit 0eea6d0 into getsops:main Jun 20, 2026
15 checks passed

BrewTestBot mentioned this pull request Jun 30, 2026

sops 3.13.2 Homebrew/homebrew-core#290535

Merged

Uh oh!

Conversation

s3onghyun commented Jun 18, 2026

Problem

Fix

Testing

Uh oh!

felixfontein commented Jun 18, 2026

Uh oh!

felixfontein commented Jun 18, 2026

Uh oh!

s3onghyun commented Jun 19, 2026

Uh oh!

felixfontein Jun 19, 2026

Choose a reason for hiding this comment

Uh oh!

s3onghyun commented Jun 19, 2026

Uh oh!

s3onghyun commented Jun 19, 2026

Uh oh!

s3onghyun commented Jun 19, 2026

Uh oh!

felixfontein left a comment

Choose a reason for hiding this comment

Uh oh!

felixfontein Jun 20, 2026

Choose a reason for hiding this comment

Uh oh!

felixfontein Jun 20, 2026

Choose a reason for hiding this comment

Uh oh!

s3onghyun commented Jun 20, 2026

Uh oh!

felixfontein left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants