fix: mask secrets in stored step outputs - #2861

Closes #2860.

What

Status masking listed node fields one by one and skipped StepOutputsValue, which arrived later (#2296). A secret a step published was stored in plain text in status.jsonl, while the log and the other output fields were masked. This applies to every producer of that field: DAGU_OUTPUT_FILE, executor-declared outputs (outputs.write, actions, dag.run, browser, git), captured stdout.outputs, and build outputs.

With the repro from the issue:

before: stepOutputsValue: {"token":"s3cr3t-value-123"}
after:  stepOutputsValue: {"token":"*******"}

No file under the Dagu home contains the secret after the run.

Retry and resume

Retry, step retry, approval/human-task/push-back resume, --outputs-from, and edit-retry all rebuild reused steps from stored status. Since stored outputs now hold the mask, a reused output that contained a secret reads as *******. This matches what output: VAR and OutputsValue already did.

Steps within the same run still exchange the real value, because masking only touches the stored snapshot. Spec 069 now states both rules, and conformance pins them.

To use a secret after an approval or on retry, read it directly ($API_TOKEN) in the step that needs it, not through another step's output.

Changes

• Mask StepOutputsValue in stored run status.
• Human-task outputs stay as entered. They are operator input, stored next to HumanTaskInput, which is also unmasked, so masking would hide nothing. A resumed run reads them back. The coordinator's exact-match check on them (preservesCompletedManualActions) is also unaffected. This follows the precedent of agent-session answers.
• Keep stored output JSON valid. A secret matching a JSON token, such as a digit-only secret inside a number, made plain replacement produce invalid JSON. A retry then failed to resolve every output of that step (unknown binding), sub-DAG outputs were dropped, and --output with --outputs-from failed to read carried outputs. Such a document is now masked value by value, including numbers, booleans and null that hold a secret. The repaired document is kept only when it holds no secret text; otherwise the plainly replaced text is stored, so the repair never masks less than plain replacement. A document that plain replacement keeps valid is stored exactly as before. This applies to OutputsValue and StepOutputsValue.
• Mask JSON-escaped secrets. Stored outputs are JSON, so a secret containing ", \, a control character, or <, >, & appeared only in escaped form and never matched. The status masker now also masks each secret's JSON-escaped forms, with and without HTML escaping. Log masking and outputs.json masking are unchanged.
• Spec 069 lists published outputs among the masked status fields and states the retry/resume rule.

Known limits

• Runs stored before this change keep their plaintext. They are not rewritten.
• A file secret with a trailing newline never matches a single-line output value. That is existing provider behavior.
• JSON encoded twice (type: json output values) or written by non-Go encoders is only partly covered.
• A secret that spans JSON values, such as one containing ",", cannot be removed while keeping the document valid. That document is stored masked but not valid JSON, so a reuse of that step's outputs fails to resolve instead of reading the secret.

Tests

• status_masking_test.go:
  • StepOutputsValue is masked and the live value is untouched.
  • Human-task outputs are kept.
  • A number, boolean or null secret leaves valid JSON.
  • A secret spanning JSON values is never written back.
  • An escaped secret is masked under both encoders.
• agent_test.go: TestAgent_StepOutputSecretMasking runs a DAG. A later step receives the real value, and the stored status holds only the mask.
• Conformance spec069_secrets: TestStepOutputMaskedOnRetry runs start, then retry. The same run reads the real value, and the retried step reads *******. It covers a plain secret and one that JSON escapes.
  • Against a pre-fix binary, both cases read real on retry.
  • Against a binary without the escaping commit, only the escaped case fails.
• Ran locally:
  • go test -race on internal/runtime/..., internal/humantask/..., internal/service/coordinator/..., internal/intake/..., internal/proto/....
  • Targeted internal/intg.
  • Conformance for spec069, spec012, spec031, spec070 and cli.
  • make check.