feat(config): coerce whole-value ${VAR} JSON arrays/objects in env substitution by hcnode · Pull Request #95603 · openclaw/openclaw

hcnode · 2026-06-21T18:00:41Z

Closes #

What Problem This Solves

Resolves a problem where array or object config cannot be supplied through an environment variable. Config env substitution (resolveConfigEnvVars) only ever returns strings, so a value written as "${VAR}" resolves to the env var's raw text. For config keys that expect an array or object (allowlists, id lists, structured limits), operators who want to keep that value out of the committed config file and inject it from the environment end up with a string where the schema expects an array/object, which fails validation.

Why This Change Was Made

This adds an explicit opt-in modifier: ${VAR:json}. Only this form coerces — when a value is exactly one ${VAR:json} reference and the resolved text is a JSON array or object literal, it is parsed into a real array/object. Plain ${VAR} always stays a string (so existing string fields are never affected), and embedded references ("prefix-${VAR:json}"), multi-var strings, scalars, JSON primitives, and malformed JSON keep their string result. Unknown modifiers (${VAR:foo}) are left as literal placeholders.

The earlier revision of this PR coerced any whole-value ${VAR} globally; review correctly flagged two compatibility problems, both now addressed:

Schema-blind coercion (fixed): coercion is now opt-in via ${VAR:json}, so string-typed fields such as env.vars (z.record(z.string(), z.string())) keep their string value even when the env value looks like JSON.
Config write-back preservation (fixed): restoreEnvVarRefs (env-preserve.ts) now restores a sole ${VAR:json} template when the incoming value is the env-resolved array/object, so the authored env reference survives config round-trips instead of being inlined into openclaw.json. The substitution and preservation paths share the same token helpers to avoid grammar drift.

User Impact

Operators can inject array/object config through environment variables by opting in with ${VAR:json}. For example, with ALLOW_FROM='["alice","bob"]', a config value of "${ALLOW_FROM:json}" resolves to the array ["alice","bob"]. Plain ${VAR} behavior, escaping ($${VAR}), and missing-var handling are unchanged, and authored ${VAR:json} references are preserved across config writes. No migration is required.

Evidence

Live terminal output from running the config loader (resolveConfigEnvVars) and the write-back path (restoreEnvVarRefs) from source via node, against a config file with allowFrom: "${ALLOW_FROM:json}" (opt-in) and a plain string-only env.vars.TAGS: "${TAGS}", with ALLOW_FROM='["alice","bob"]' and TAGS='["x","y"]'. Console output:

[1] resolved config:
{
  "channels": { "discord": { "allowFrom": [ "alice", "bob" ] } },
  "env": { "vars": { "TAGS": "[\"x\",\"y\"]" } }
}
opt-in allowFrom isArray: true
plain TAGS stays string: true
[2] write-back (what gets persisted to disk):
{
  "channels": { "discord": { "allowFrom": "${ALLOW_FROM:json}" } },
  "env": { "vars": { "TAGS": "${TAGS}" } }
}
allowFrom ref preserved: true

This demonstrates all three guarantees: (1) ${ALLOW_FROM:json} coerces to a real array; (2) plain ${TAGS} stays a string even though its value is JSON-looking (string fields are safe); (3) on write-back the authored ${ALLOW_FROM:json} reference is restored rather than inlined.

Focused tests: src/config/env-substitution.test.ts and src/config/env-preserve.test.ts (92 passed total), covering opt-in coercion, plain-${VAR} passthrough, embedded/scalar/malformed/primitive passthrough, unknown-modifier literal passthrough, and ${VAR:json} write-back round-trip (restore + caller-changed-value cases). pnpm tsgo is clean.

xuwei-xy · 2026-06-21T18:01:05Z

Good catch! This looks like a valid bug report.

clawsweeper · 2026-06-21T18:02:19Z

Codex review: needs maintainer review before merge. Reviewed July 11, 2026, 2:07 PM ET / 18:07 UTC.

Summary
The PR adds an opt-in ${VAR:json} placeholder that parses whole-value environment variables into JSON arrays or objects, preserves authored references during config write-back, and adds focused tests and public documentation.

PR surface: Source +110, Tests +122, Docs +62. Total +294 across 7 files.

Reproducibility: not applicable. current main's ordinary ${VAR} string substitution behaves as defined, while this PR adds a new opt-in structured-value capability rather than repairing a violated existing contract.

Review metrics: 1 noteworthy metric.

Public config grammar: 1 modifier added. ${VAR:json} creates a new compatibility obligation across config loading, includes, persistence, and documentation.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] Merging establishes ${VAR:json} as a permanent public config grammar and write-back compatibility contract that future loaders, include handling, migrations, and documentation must preserve.
[P1] The parser is deliberately schema-blind after explicit opt-in; placing ${VAR:json} on a string-typed field can still produce a validation error, making that failure model part of the supported product behavior.

Maintainer options:

Close pending sponsorship (recommended)
Close the technically credible PR until a maintainer explicitly owns the new public config-language and upgrade contract.
Sponsor the config contract
Accept the syntax intentionally and own compatibility across loading, includes, persistence, migrations, and documentation before merging.

Next step before merge

This is a product-direction close; no automated repair is appropriate unless a maintainer first sponsors the new public config grammar.

Maintainer decision needed

Question: Should OpenClaw adopt ${VAR:json} as the supported generic syntax for environment-backed array and object config values?
Rationale: Code review supports the implementation, but it cannot authorize a permanent config-language modifier and persistence compatibility obligation without maintainer intent.
Likely owner: steipete — The available review and history evidence points to him as the strongest decision owner across the config parser, preservation, and IO contract.
Options:
- Close pending sponsorship (recommended): Close this technically sound proposal and reopen or request a refreshed implementation if a maintainer later sponsors the config-language direction.
- Sponsor and adopt: Explicitly accept ${VAR:json} as a public contract and proceed through normal exact-head merge validation.
- Keep grammar unchanged: Decline a generic modifier and retain string substitution or narrower field-specific mechanisms for structured values.

Security
Cleared: The patch adds bounded local JSON.parse data coercion without new dependencies, workflows, permission changes, downloads, secret exposure, or executable evaluation.

Review details

Best possible solution:

Keep the implementation design and proof available, but adopt it only after a maintainer explicitly sponsors the public config-language contract; a sponsored version should retain this shared parser/preservation path, focused tests, and public documentation.

Do we have a high-confidence way to reproduce the issue?

Not applicable: current main's ordinary ${VAR} string substitution behaves as defined, while this PR adds a new opt-in structured-value capability rather than repairing a violated existing contract.

Is this the best way to solve the issue?

Yes in implementation shape: explicit whole-value array/object coercion is narrower and safer than implicit schema-blind parsing, but adopting the syntax requires maintainer product approval.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against d4adddbe9c7a.

Label changes

Label changes:

add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.

Label justifications:

P2: This is a bounded, well-implemented config feature with limited blast radius but a meaningful public-contract decision.
merge-risk: 🚨 compatibility: Merging adds permanent user-authored config syntax and write-back semantics that future releases and upgrade paths must preserve.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body contains after-fix live output from the config loader and write-back path demonstrating structured coercion, ordinary-string safety, and authored-reference preservation.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body contains after-fix live output from the config loader and write-back path demonstrating structured coercion, ordinary-string safety, and authored-reference preservation.

Evidence reviewed

PR surface:

Source +110, Tests +122, Docs +62. Total +294 across 7 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	2	135	25	+110
Tests	2	122	0	+122
Docs	3	62	0	+62
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	7	319	25	+294

What I checked:

Opt-in parser boundary: The exact head recognizes only the :json modifier, limits coercion to a sole whole-value token, and accepts only arrays or plain objects; ordinary ${VAR}, embedded references, primitives, malformed JSON, and unknown modifiers remain strings or literals. (src/config/env-substitution.ts:54, 18be59d50915)
Write-back preservation: The preservation path reuses substitution helpers and restores an authored ${VAR:json} reference only when the incoming structure still deep-equals the environment-resolved array or object. (src/config/env-preserve.ts:748, 18be59d50915)
Focused regression coverage: Tests cover array/object coercion, ordinary string safety, embedded and malformed values, escaping, unknown modifiers, missing-value modes, structured restoration, and caller-changed values. (src/config/env-substitution.test.ts:402, 18be59d50915)
Public contract documented: The guide and reference document the opt-in, whole-value, array/object-only grammar, string-field safety, escaping, missing and unknown modifier behavior, and write-back preservation, resolving the earlier review finding. Public docs: docs/gateway/configuration.md. (docs/gateway/configuration.md:696, 18be59d50915)
Current-main preservation ancestry: The contributor's head contains the recent current-main preservation implementation, so the earlier concern about omitting newer array-identity and escaped-reference safeguards no longer applies. (src/config/env-preserve.ts:1, f73c5541fe56)
Real behavior proof: The PR body records a human-run source invocation where ${ALLOW_FROM:json} becomes an array, ordinary ${TAGS} remains a string, and write-back restores both authored references. (18be59d50915)

Likely related people:

steipete: The prior completed history review identified Peter Steinberger as having the strongest recent involvement across the parser, preservation, and config IO surfaces defining this public contract. (role: likely product decision owner; confidence: medium; files: src/config/env-substitution.ts, src/config/env-preserve.ts, src/config/io.ts)
xingzhou: Commit f73c554 provides current-main provenance for the recent config preservation implementation inherited by this PR head. (role: recent area contributor; confidence: high; commits: f73c5541fe56; files: src/config/env-preserve.ts, src/config/env-preserve.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (2 earlier review cycles)

reviewed 2026-06-29T04:57:18.485Z sha 351d96d :: found issues before merge. :: [P2] Document the new config modifier
reviewed 2026-07-11T17:21:28.197Z sha f851883 :: needs maintainer review before merge. :: none

resolveConfigEnvVars only ever returned strings, so a config value of "${VAR}" resolved to the env var's raw text. Array/object config keys (allowlists, id lists, structured limits) could not be injected via env vars: the schema expected an array/object but received a string. Add an explicit opt-in modifier ${VAR:json}: when a value is exactly one ${VAR:json} reference and the resolved text is a JSON array or object literal, parse it into a real array/object. Plain ${VAR} always stays a string, so existing string-typed fields (e.g. env.vars) are unaffected. Embedded refs, multi-var strings, scalars, JSON primitives, malformed JSON, and unknown modifiers all keep their string/literal result. The token grammar preserves the verbatim braces content (`inner`) so escaped $${VAR:json} stays literal and unresolved ${VAR:json} placeholders keep the modifier in onMissing mode. The same `:json` handling is mirrored in env-preserve's resolver. Preserve authored ${VAR:json} references across config write-back: restoreEnvVarRefs restores a sole ${VAR:json} template when the incoming value deep-equals the env-resolved array/object, instead of inlining the resolved structure into openclaw.json. Substitution and preservation share the same token semantics to avoid grammar drift. Adds tests for opt-in coercion, plain-${VAR} passthrough, escaped $${VAR:json}, onMissing modifier preservation, and write-back round-trip.

…n-array-coercion

…fier Adds guide + reference docs for the opt-in ${VAR:json} coercion: whole-value opt-in, array/object-only parsing, string-field safety, escaping/missing/unknown modifier behavior, and write-back preservation. Closes the ClawSweeper docs blocker for the new config syntax.

hcnode · 2026-07-11T17:51:22Z

Updated: added the public docs the review asked for, in both docs/gateway/configuration.md and docs/gateway/configuration-reference.md. They document the ${VAR:json} modifier: opt-in whole-value coercion, array/object-only parsing, string-field safety (plain ${VAR} unchanged), escaping ($${VAR:json}), missing-var and unknown-modifier behavior, and write-back preservation. Also merged latest main.

src/config/env-substitution.test.ts + src/config/env-preserve.test.ts: 94 passed.

@clawsweeper re-review

clawsweeper · 2026-07-11T17:51:24Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Failed
Detail: The targeted re-review did not finish cleanly. Check the workflow run for details.
Run: https://github.com/openclaw/clawsweeper/actions/runs/29162427531
Updated: 2026-07-11T18:00:39.339Z

openclaw-barnacle · 2026-07-26T04:34:21Z

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

openclaw-barnacle Bot added the size: S label Jun 21, 2026

hcnode force-pushed the feat/config-env-json-array-coercion branch from 63f878e to bc80063 Compare June 21, 2026 18:38

openclaw-barnacle Bot added size: M and removed size: S labels Jun 21, 2026

hcnode force-pushed the feat/config-env-json-array-coercion branch from bc80063 to c344775 Compare June 21, 2026 18:59

totobusnello mentioned this pull request Jun 21, 2026

fix(agents): preserve streamed assistant text when Claude CLI result event is empty #90450

Merged

3 tasks

hcnode added 2 commits June 21, 2026 20:33

Merge branch 'main' into feat/config-env-json-array-coercion

4bb3386

Merge branch 'main' into feat/config-env-json-array-coercion

351d96d

zhiling-chen-20230331 added 2 commits July 11, 2026 10:13

Merge remote-tracking branch 'upstream/main' into feat/config-env-jso…

4eb855c

…n-array-coercion

openclaw-barnacle Bot added docs Improvements or additions to documentation gateway Gateway runtime labels Jul 11, 2026

docs(config): regenerate docs_map for new ${VAR:json} reference heading

18be59d

clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jul 11, 2026

openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jul 26, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(config): coerce whole-value ${VAR} JSON arrays/objects in env substitution#95603

feat(config): coerce whole-value ${VAR} JSON arrays/objects in env substitution#95603
hcnode wants to merge 6 commits into
openclaw:mainfrom
hcnode:feat/config-env-json-array-coercion

hcnode commented Jun 21, 2026 •

edited

Loading

Uh oh!

xuwei-xy commented Jun 21, 2026

Uh oh!

clawsweeper Bot commented Jun 21, 2026 •

edited

Loading

Uh oh!

hcnode commented Jul 11, 2026

Uh oh!

clawsweeper Bot commented Jul 11, 2026 •

edited

Loading

Uh oh!

openclaw-barnacle Bot commented Jul 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

hcnode commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

xuwei-xy commented Jun 21, 2026

Uh oh!

clawsweeper Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hcnode commented Jul 11, 2026

Uh oh!

clawsweeper Bot commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openclaw-barnacle Bot commented Jul 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hcnode commented Jun 21, 2026 •

edited

Loading

clawsweeper Bot commented Jun 21, 2026 •

edited

Loading

clawsweeper Bot commented Jul 11, 2026 •

edited

Loading