Skip to content

feat(redisotel): truncate statement span attributes - #4114

Open
ronitgupta138 wants to merge 11 commits into
redis:masterfrom
ronitgupta138:feat/redisotel-truncate-statement
Open

ronitgupta138 wants to merge 11 commits into
redis:masterfrom
ronitgupta138:feat/redisotel-truncate-statement

Conversation

@ronitgupta138

@ronitgupta138 ronitgupta138 commented Oct 16, 2026 •

Copy link
Copy Markdown

Summary

Fixes #4092.

Truncates db.statement span attributes by default in extra/redisotel to prevent large command payloads from being copied and pushed into telemetry.

Details

  • Truncates command arguments exceeding the statement max length limit (DefaultDBStatementMaxLen = 512) with a summary suffix …(+N bytes).
  • Truncation occurs while building the statement string so large payloads (e.g. 1MB) are never copied.
  • Implements a total cap for the overall statement, including multi-command pipelines.
  • Adds WithDBStatementMaxLen(maxLen int) TracingOption allowing callers to customize the truncation threshold, or opt in to full statements by passing maxLen <= 0.
  • Preserves credential redaction (AUTH, HELLO ... AUTH, CONFIG SET requirepass, ACL SETUSER, MIGRATE ... AUTH, SENTINEL SET auth-pass) across both truncated and untruncated statements.
  • Extracts pipeline summary command names in pipelineSummary without copying or allocating argument strings.

Test Plan

  • extra/redisotel/statement_test.go verifies:
    • Default truncation on 1MB payloads and boundary values.
    • Custom threshold adjustments with WithDBStatementMaxLen.
    • Full data opt-in with WithDBStatementMaxLen(0).
    • Credential redaction with both small and large passwords.
    • Pipeline command truncation and total statement cap.
    • Large binary byte slice hex-encoding and truncation.
  • go test -race -v ./extra/redisotel/... passes cleanly.

Note

Medium Risk
Default telemetry behavior changes (truncated db.statement) and touches sensitive-command formatting, though redaction and limits are heavily tested.

Overview
Adds length-bounded Redis command rendering in extra/rediscmd (CmdStringLimit, CmdsStringLimit, CmdsSummary) so large arguments are truncated with …(+N bytes) without copying megabyte payloads, while keeping existing credential redaction and extending it (ACL rule pointers, custom Stringer types, panicking String() recovery).

extra/redisotel now defaults db.statement to a 512-byte budget via DefaultDBStatementMaxLen and WithDBStatementMaxLen, wiring tracing hooks through buildCmdStatement / buildPipelineStatement and deriving pipeline span names from CmdsSummary instead of formatting every argument. Callers can pass maxLen <= 0 to retain full statements (existing long-statement tests updated accordingly).

Reviewed by Cursor Bugbot for commit 29a2863. Bugbot is set up for automated code reviews on this repo. Configure here.

Truncate db.statement span attributes by default to avoid pushing large
command payloads into telemetry. Arguments exceeding the limit are
cut to a prefix with a summary marker '…(+N bytes)', and pipeline
statements are capped. Full data remains opt-in via
WithDBStatementMaxLen(0). Credential arguments remain redacted.

Closes redis#4092
Copilot AI balanced review requested due to automatic review settings October 16, 2026 08:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-11T15:28:03.553575Z 29a2863 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 004c76283c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extra/redisotel/statement.go Outdated
b = append(b, redactedArg...)
continue
}
b = appendArgTruncated(b, arg, maxLen)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Enforce the remaining statement budget per argument

When a command has a long but sub-limit key followed by a max-sized value (for example, SET with a 500-byte key and 512-byte value under the default), the pre-append check still passes and this call grants the value another full maxLen; the resulting db.statement is about 1 KiB (and binary data can be larger after hex encoding), so WithDBStatementMaxLen does not provide its documented overall statement cap. Pass only the remaining budget, accounting for encoded output and markers, when appending.

Useful? React with 👍 / 👎.

Comment thread extra/redisotel/statement.go Outdated
_, cmdsString := rediscmd.CmdsString(cmds)
return cmdsString
}
b := make([]byte, 0, 64*len(cmds))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bound pipeline buffer allocation by the limit

When tracing a large pipeline with truncation enabled, this allocation reserves 64 * len(cmds) bytes before the loop stops near the configured cap; a 100,000-command pipeline therefore creates a 6.4 MB temporary buffer to retain only roughly 512 bytes. This makes the instrumentation's memory and GC cost scale with the entire pipeline despite the new cap, so bound the initial capacity by the configured maximum.

Useful? React with 👍 / 👎.

…e buffer allocation (redis#4092)

- Subtract written statement bytes from maxLen before appending arguments to strictly enforce overall statement cap
- Account for 2x hex expansion on non-printable binary arguments when computing truncation thresholds
- Cap pipeline buffer allocation to maxLen to avoid allocating multi-megabyte buffers on large pipelines
@ronitgupta138

Copy link
Copy Markdown
Author

Addressed the review findings in commit b77fac2:

  1. Per-Argument Statement Budget: Updated appendCmdTruncated to pass remaining := maxLen - len(b) instead of granting each argument another full maxLen, ensuring single commands with multiple large arguments strictly adhere to the overall configured statement limit.
  2. Binary Hex-Encoding Expansion: Handled 2x expansion for non-printable binary byte slices/strings so that hex-encoded representations do not exceed the remaining budget before truncation.
  3. Pipeline Buffer Cap: Capped the initial capacity of buildPipelineStatement buffer to maxLen + 64 instead of allocating 64 * len(cmds) unconditionally, preventing large pipelines from reserving megabytes of memory under truncation.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b77fac2edb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extra/redisotel/statement.go Outdated
Comment on lines +231 to +235
if isHex && limit > 0 {
byteLimit = limit / 2
}
if byteLimit > 0 && len(v) > byteLimit {
b = appendUTF8String(b, v[:byteLimit])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat a zero binary budget as exhausted

With truncation enabled, the new remaining-budget path can pass limit == 1 here; integer division then sets byteLimit to zero, and the byteLimit > 0 guard skips truncation so the fallthrough hex-encodes the entire argument. For example, at the default 512-byte limit, SET plus a 506-byte key leaves one byte for a 1 MiB non-printable value and produces roughly 2 MiB of db.statement, so this still defeats the cap and recreates the large telemetry allocation; treat zero as an exhausted encoded budget rather than as unlimited.

Useful? React with 👍 / 👎.

Comment thread extra/redisotel/statement.go Outdated
Comment on lines +215 to +216
bytesVal := []byte(v)
isHex := !isSimple(bytesVal)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bound classification work to the retained prefix

When truncation is enabled for a large printable string, such as a multi-megabyte JSON value, isSimple(bytesVal) walks the entire argument before appendArgTruncated retains only the first remaining-budget bytes. Because this hook runs synchronously before the Redis operation, default tracing still adds O(payload size) work despite the 512-byte limit, so inspect only the retained prefix (or otherwise bound classification work) rather than the discarded suffix.

Useful? React with 👍 / 👎.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread extra/redisotel/statement.go Resolved
Comment thread extra/redisotel/statement.go Resolved
…ard zero binary budget (redis#4092)

- Restrict isSimple check to at most limit bytes to avoid O(payload) work on large strings/bytes
- Avoid full []byte(v) allocations by slicing string arguments directly before encoding
- Guard byteLimit <= 0 when remaining budget is 1 byte, summarizing without hex-dumping entire payload
@ronitgupta138

Copy link
Copy Markdown
Author

Addressed the review findings in commit 6e59a1b:

  1. Bounded Inspection & Zero Full-Payload Copies:
    • Replaced full []byte(v) string conversions with direct slicing on the string prefix (v[:charLimit]), eliminating multi-megabyte heap allocations for large strings under truncation.
    • Bounded character classification to checkLen := min(len(s), limit), ensuring isSimpleString never iterates over discarded suffixes on large strings/JSON payloads (converting (N)$ inspection work to (\text{limit})$).
  2. Guarded Budget Exhaustion at 1 Byte:
    • Explicitly handled byteLimit <= 0 when remaining budget is 1 byte for non-printable binary data, emitting the truncated summary marker (…(+N bytes)) rather than falling through to hex-dump the full multi-megabyte payload.
  3. Tests Added:
    • Added unit test verifying that a 1-byte remaining budget on a 1MB binary payload summarizes without dumping hex characters.
    • Added unit test verifying that a 5MB JSON string truncates cleanly without full string copies.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e59a1b106

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extra/redisotel/statement.go Outdated
case string:
return v
case []byte:
return string(v)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid copying byte-slice rules before truncation

When ACL SETUSER receives a large rule as a []byte (for example, a multi-megabyte >password), secretArgs calls argString for every rule and this conversion copies the entire slice before the rule is redacted; the later truncation therefore keeps at most maxLen but the tracing hook still incurs a payload-sized allocation. Inspect the byte slice directly or compare it without converting the whole value to a string.

Useful? React with 👍 / 👎.

Comment thread extra/redisotel/statement.go Outdated
Comment on lines +146 to +147
if maxLen > 0 && bufCap > maxLen+64 {
bufCap = maxLen + 64

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard the pipeline capacity addition from overflow

When a caller supplies WithDBStatementMaxLen a value above math.MaxInt-64, maxLen+64 wraps negative, this branch assigns a negative bufCap, and the subsequent make panics before the wrapped Redis hook is called. Compute this cap with overflow-safe or saturating arithmetic so a valid positive configuration cannot crash pipeline processing.

AGENTS.md reference: AGENTS.md:L117-L121

Useful? React with 👍 / 👎.

…rsion on byte-slice ACL rules (redis#4092)

- Guard pipeline buffer capacity calculation against integer overflow for extreme maxLen values
- Inspect ACL rules directly on []byte slices without allocating temporary strings
@ronitgupta138

Copy link
Copy Markdown
Author

Addressed the review findings in commit b4dd5f9:

  1. Zero-Allocation Byte-Slice ACL Inspection: Replaced argString(args, i) string conversion in secretArgs with hasSecretRulePrefix, inspecting []byte rules directly by checking prefix bytes (>, <, #, !) without converting byte slices to strings.
  2. Overflow-Safe Buffer Capacity: Updated pipeline initial buffer calculation to guard against potential integer overflow near math.MaxInt.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b4dd5f9a4f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extra/redisotel/statement.go Outdated
case time.Time:
return v.AppendFormat(b, time.RFC3339Nano)
default:
s := fmt.Sprint(v)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid materializing custom arguments before truncation

When a valid command argument is a large custom value (for example, a named []byte implementing encoding.BinaryMarshaler, which internal/proto.Writer.WriteArg accepts), fmt.Sprint(v) constructs a representation of the entire payload before limit is applied. A 1 MiB binary-marshaler slice can therefore allocate a multi-megabyte decimal string synchronously in the tracing hook despite the default 512-byte setting, defeating the new allocation bound; handle supported custom values without formatting the discarded suffix.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit 59d9d7d: added explicit support for encoding.BinaryMarshaler, net.IP, and pointer arguments in appendArgTruncated, directly bounding the binary representation to the retained prefix without materializing full formatted strings via fmt.Sprint.

redis#4092)

- Handle encoding.BinaryMarshaler, net.IP, and pointer arguments in appendArgTruncated without falling through to fmt.Sprint
- Bound custom binary arguments to byteLimit prefix without materializing full formatted slice
- Add unit test coverage for large custom BinaryMarshaler payloads and pointer arguments

Signed-off-by: Ronit Gupta <[email protected]>
@ronitgupta138

Copy link
Copy Markdown
Author

Addressed the review findings in commit 59d9d7d:

  1. Zero-Materialization for Custom Binary Arguments: Added native support for encoding.BinaryMarshaler and net.IP in appendArgTruncated, delegating directly to appendBytesTruncated so large custom payloads retain only the prefix bounded by the remaining budget without materializing multi-megabyte decimal slice strings via fmt.Sprint.
  2. Pointer Support: Added proper dereferencing for common pointer argument types (*string, *int, etc.).
  3. Tests Added: Added TestTracingHook_StatementTruncation_CustomTypes verifying that custom types implementing encoding.BinaryMarshaler with multi-megabyte payloads truncate cleanly without invoking fallback slice string formatters.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 87dc67b9e4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extra/redisotel/statement.go Outdated
Comment on lines +258 to +259
case encoding.BinaryMarshaler:
data, err := v.MarshalBinary()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Match time values before BinaryMarshaler

When tracing an HSET struct with a nil *time.Time field (a value supported by the command writer), the pointer satisfies encoding.BinaryMarshaler and reaches this call before the nil-aware *time.Time branch, so Time.MarshalBinary panics even though internal/proto/writer.go:189-210 would serialize it as the zero time; non-nil time values are likewise logged as binary hex instead of the RFC3339 value sent to Redis. Move the time cases before the interface dispatch so instrumentation does not crash or misreport otherwise valid commands.

AGENTS.md reference: AGENTS.md:L117-L123

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit a2dd0ee: moved time.Time, *time.Time, time.Duration, and *time.Duration before any custom type interface dispatch. Non-nil timestamps format as RFC3339Nano matching proto.Writer, and nil *time.Time pointers safely format as <nil> without dereferencing or panicking.

Comment thread extra/redisotel/statement.go Outdated
}
return appendBytesTruncated(b, *v, limit)
case encoding.BinaryMarshaler:
data, err := v.MarshalBinary()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid fully marshaling truncated custom values

When tracing a large encoding.BinaryMarshaler whose method builds a fresh result, this call constructs the entire payload before appendBytesTruncated can apply the 512-byte budget, and internal/proto/writer.go:205-210 invokes it again to send the request. The fresh evidence after the earlier custom-argument fix is that MarshalBinary accepts no limit and returns the complete byte slice, so a multi-megabyte value still adds a payload-sized tracing allocation; use a bounded representation that does not fully marshal it in the hook.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit a2dd0ee: custom types now inspect underlying []byte / string slices directly via reflection to avoid payload-sized allocations in tracing hooks.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread extra/redisotel/statement.go Resolved
…ll marshaling (redis#4092)

- Move time.Time, *time.Time, and time.Duration cases before custom type reflection to ensure timestamps serialize as RFC3339Nano and nil pointers format as <nil> without panicking
- Use reflect-based underlying byte slice extraction for custom types to truncate without full payload marshaling allocations
- Add unit tests verifying nil *time.Time safety and non-nil time.Time RFC3339Nano formatting

Signed-off-by: Ronit Gupta <[email protected]>
@ronitgupta138

Copy link
Copy Markdown
Author

Addressed the review findings in commit a2dd0ee:

  1. Prioritize Time & Pointer Serialization: Placed time.Time, *time.Time, time.Duration, and *time.Duration before any custom type dispatch, preventing time.Time from matching generic binary marshalers. Timestamps format cleanly in standard RFC3339Nano, and nil *time.Time pointers format as <nil> without dereferencing or panicking.
  2. Zero-Allocation Custom Type Slicing: Replaced full MarshalBinary() calls with zero-allocation reflection inspection of underlying byte/string slices, ensuring custom types retain only the budgeted prefix without generating multi-megabyte payloads in tracing hooks.
  3. Tests Added: Added unit tests verifying nil *time.Time safe formatting and non-nil time.Time RFC3339 formatting.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a2dd0ee7ef

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extra/redisotel/statement.go Outdated
Comment on lines +405 to +406
if val.Kind() == reflect.String {
return appendStringTruncated(b, val.String(), limit)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve redaction supplied by named Stringer arguments

When a named string argument implements both fmt.Stringer (returning a redacted value) and encoding.BinaryMarshaler (so internal/proto.Writer.WriteArg accepts it), this reflection branch reads the underlying string directly instead of using the safe String result. The parent path through rediscmd.CmdString used fmt.Sprint for such named types, so the default max-length path now exports secrets that were previously masked in db.statement; keep a bounded representation without exposing the underlying value.

Useful? React with 👍 / 👎.

Comment thread extra/redisotel/statement.go Outdated
}
}

s := fmt.Sprint(v)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bound formatting of struct-backed custom arguments

When a valid argument is a struct-backed encoding.BinaryMarshaler containing a large byte slice, internal/proto.Writer.WriteArg accepts it but this fallback renders the entire struct and slice before appendStringTruncated applies the 512-byte budget. The fresh evidence after the earlier custom-argument fixes is that the new reflection branches only bound underlying string/slice kinds; a 1 MiB marshaler struct still creates a multi-megabyte decimal string synchronously in tracing, so use a bounded placeholder or representation instead of formatting the whole value.

Useful? React with 👍 / 👎.

Comment thread extra/redisotel/statement.go Outdated

if err := cmd.Err(); err != nil {
b = append(b, ": "...)
b = append(b, err.Error()...)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Truncate command errors within the statement budget

When a command enters Process with Err() already set to a long message (callers can do this through the public Cmder.SetErr API), this append bypasses all argument budgeting. A 1 MiB pre-set error therefore still creates and copies a 1 MiB db.statement under the default 512-byte limit, and pipelines likewise grow beyond their cap; append only the error prefix that fits the remaining budget.

Useful? React with 👍 / 👎.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread extra/redisotel/statement.go Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33fa0b1b99

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extra/redisotel/statement.go Outdated
}
return append(b, v.String()...)
case int:
return strconv.AppendInt(b, int64(v), 10)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Truncate scalar arguments to the remaining budget

When WithDBStatementMaxLen is smaller than a scalar's textual form, the numeric, time, and bool branches append the entire value and ignore limit; for example, INCRBY key math.MaxInt64 with maxLen == 16 has five bytes left here but emits all 19 digits, producing a 30-byte statement with no truncation marker. The caller now passes the correct remaining budget, but these branches still bypass it, so scalar formatting should also use a bounded append path.

Useful? React with 👍 / 👎.

@mdreizin

mdreizin commented Nov 4, 2026

Copy link
Copy Markdown

Thanks for picking this up. We hit #4092 in our service: cached values of about 1 MB, hex-encoded into db.statement, caused memory spikes, and OTLP exports were rejected for exceeding 4 MiB. Building the shortened statement directly, without the full string first, is exactly the right fix. A few ideas from our use case:

1. Limit each argument, not the whole statement.
appendCmdTruncated uses remaining = maxLen - len(b), and buildPipelineStatement stops with \n… once the shared budget is used up. So one large value cuts off everything after it. In a typical cache write, the useful part comes after the value:

set cache:page:1 <1 MB value> ex 86400
sadd cache:tag:a cache:page:1
PEXPIRE cache:tag:a 86400000 NX
PEXPIRE cache:tag:a 86400000 GT

With a per-argument limit, every command keeps its name, keys and options:

set cache:page:1 48656c...(+1048573 bytes) ex 86400
sadd cache:tag:a cache:page:1
PEXPIRE cache:tag:a 86400000 NX
PEXPIRE cache:tag:a 86400000 GT

2. Make the total cap optional and off by default.
The OTel SDK attribute limits (OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT) already bound the total size. If a statement cap is wanted, it could be a separate option, such as WithDBStatementTotalMaxLen.

3. Expose formatting as options, so users can pick a strategy.
For example:

// Formats one argument. The default shows at most maxLen characters, then "...(+N bytes)".
WithDBStatementArgFormatter(func(b []byte, arg any) []byte)
DefaultDBStatementArgFormatter(maxLen int)

// Or whole statements, for full control.
WithDBStatementFormatter(func(cmd redis.Cmder) string)
WithPipelineDBStatementFormatter(func(cmds []redis.Cmder) string)

An argument formatter that is handed the bytes would keep credential redaction in redisotel and leave only the argument rendering to the user. Strategies could then include the size only (<N bytes>), a prefix plus size, keys only, or the [N other arguments] form used by @opentelemetry/instrumentation-ioredis.

4. Don't build the statement just for the span name.
ProcessPipelineHook always calls rediscmd.CmdsString(cmds), because the span name uses its summary. That happens even with WithDBStatement(false), so the full multi-MB string is still built and thrown away. Computing the summary from cmd.FullName() alone, as pipelineSummary in this PR does, avoids that. A WithSpanNameFormatter option would make it customizable too.

Our current workaround, in case it helps others. go-redis sends an encoding.BinaryMarshaler argument through MarshalBinary(), while rediscmd.appendArg formats unknown types with fmt.Sprint, which calls String(). So we wrap stored values:

type value struct {
	b      []byte
	format func([]byte) string
}

func (v value) MarshalBinary() ([]byte, error) { return v.b, nil }
func (v value) String() string                 { return v.format(v.b) }

pipe.Set(ctx, key, value{b, format}, ttl)

Redis gets the exact bytes, and the statement shows 48656c...(+1048573 bytes). It works today but depends on internal behavior. Built-in options would be cleaner.

@ndyakov ndyakov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ronitgupta138, and apologies that this took a while: we've had a big wave of contributions lately and the review queue backed up. The truncation does what #4092 asks.

Two things before merge:

  1. Credential leak on the default path (inline 2): a named string type with a redacting String() is written raw, so SET k Secret("hunter2") becomes SET k hunter2, while master and WithDBStatementMaxLen(0) give SET k ***. Named []byte types have the same problem.
  2. Please add the limit to extra/rediscmd and call it from here instead of copying its redaction table (inline 1), so redaction stays in one place.

Smaller points are inline 3 and 4. Also please drop (#4092) from the title; squash merge adds the PR number.

Comment thread extra/redisotel/statement.go Outdated

// secretArgs marks the positions in args that hold a credential. It returns nil
// when the command carries none, which is the common case.
func secretArgs(args []interface{}) []bool {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

secretArgs, secretConfigParams, equalFoldArg, argString, appendUTF8String, isSimple and the number cases of appendArgTruncated copy extra/rediscmd/rediscmd.go. Please move the limit into rediscmd and call it from here, so redaction lives in one place. hasSecretRulePrefix and the pointer cases can move with it. argString can then use the existing zero-copy String(v) instead of string(v), which also covers the open Codex thread on line 164.

Comment thread extra/redisotel/statement.go Outdated
Comment on lines +405 to +406
if val.Kind() == reflect.String {
return appendStringTruncated(b, val.String(), limit)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reflect.Value.String() on a Kind() == String value returns the underlying string and skips the type's own String() method. With type Secret string that has String() string { return "***" } (plus MarshalBinary, so WriteArg accepts it), the default path writes the secret. The []byte-kind branch on line 409 has the same problem. When truncation is on, a fmt.Stringer should go through appendStringTruncated(b, v.String(), limit). Only a string/[]byte kind with no methods should use the kind fast path. Please add a test with a redacting Stringer.

Comment thread extra/redisotel/statement.go Outdated
}
}

s := fmt.Sprint(v)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still unbounded for struct-backed BinaryMarshaler values: fmt.Sprint formats the whole value and the result is cut afterwards. Master does the same, but the PR description says large payloads are not copied. A bounded option is a type placeholder such as <main.Blob> when truncation is on, with fmt.Sprint kept only for maxLen <= 0.

Comment thread extra/redisotel/config.go Outdated
Comment on lines +155 to +157
// WithDBStatementMaxLen specifies the maximum length for statements and arguments in the tracing hook.
// Arguments exceeding this limit are truncated with a summary marker `…(+N bytes)`.
// If maxLen <= 0, statements and arguments are not truncated (full data opt-in).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limit is one budget for the whole statement (one command, or all commands of a pipeline), and the marker and the command error text can go past it. Suggested wording:

Suggested change
// WithDBStatementMaxLen specifies the maximum length for statements and arguments in the tracing hook.
// Arguments exceeding this limit are truncated with a summary marker `…(+N bytes)`.
// If maxLen <= 0, statements and arguments are not truncated (full data opt-in).
// WithDBStatementMaxLen sets the byte budget for the db.statement attribute
// (default DefaultDBStatementMaxLen). The budget covers the whole statement:
// one command, or all commands of a pipeline. Arguments that do not fit are cut
// and get a `…(+N bytes)` marker; the marker and the command error text can go
// past the budget. If maxLen <= 0, statements are not truncated.

…iscmd

- Move statement truncation logic and length limits into extra/rediscmd/rediscmd.go
  via CmdStringLimit, CmdsStringLimit, and AppendCmdLimit.
- Prioritize fmt.Stringer to ensure custom String() redaction masks (e.g. Secret types)
  are preserved and not leaked via raw reflection.
- Bound struct-backed BinaryMarshaler representations to type placeholders when truncated.
- Use zero-copy rediscmd.String(v) in argString.
- Update WithDBStatementMaxLen docstring per maintainer review.
- Add tests for Stringer redaction and truncation in both rediscmd and redisotel.

Signed-off-by: Ronit Gupta <[email protected]>
@ronitgupta138 ronitgupta138 changed the title feat(redisotel): truncate statement span attributes (#4092) feat(redisotel): truncate statement span attributes Nov 5, 2026
@ronitgupta138

Copy link
Copy Markdown
Author

Thanks @ndyakov for the thorough review!

Addressed all points in 266fba7:

  1. Unify Redaction & Truncation in rediscmd: Moved statement truncation and length limits into extra/rediscmd (CmdStringLimit, CmdsStringLimit, and AppendCmdLimit). extra/redisotel/statement.go now delegates directly to rediscmd.
  2. Prevent Credential Leaks on Custom Types: Prioritized fmt.Stringer ahead of raw reflection so types like type Secret string with custom String() methods preserve their mask (e.g. ***) rather than exposing raw underlying strings. Added redacting Stringer test cases to both rediscmd_test.go and statement_test.go.
  3. Zero-Copy & Rule Prefix Helpers: In argString, used zero-copy String(v) from rediscmd, resolving the open Codex discussion. Moved hasSecretRulePrefix into rediscmd.
  4. Bounded BinaryMarshaler: When truncation is active (limit > 0), struct-backed values are formatted using bounded type placeholders (<pkg.TypeName>) to avoid allocating large unconstrained strings.
  5. Docstring & Title: Updated WithDBStatementMaxLen docstring with the suggested wording, and dropped (#4092) from the PR title.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3a328b93de

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extra/rediscmd/rediscmd.go Outdated
if v == nil || (reflect.ValueOf(v).Kind() == reflect.Pointer && reflect.ValueOf(v).IsNil()) {
return append(b, "<nil>"...)
}
return appendStringTruncated(b, s.String(), limit)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Contain Stringer panics inside statement rendering

When an otherwise valid encoding.BinaryMarshaler also implements fmt.Stringer and its String method panics for a particular value, this direct call lets the panic escape the built-in tracing hook before next is called; the Redis writer would only call MarshalBinary, and the previous fmt.Sprint path recovered formatter panics. Use a panic-safe rendering path so telemetry cannot abort an otherwise serializable Redis operation.

AGENTS.md reference: AGENTS.md:L117-L122

Useful? React with 👍 / 👎.

Comment thread extra/redisotel/statement.go Outdated
}

func pipelineSummary(cmds []redis.Cmder) string {
summary, _ := rediscmd.CmdsStringLimit(cmds, 0)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Build pipeline summaries without rendering arguments

When ProcessPipelineHook sees a pipeline containing a large value, this call passes 0, so CmdsStringLimit renders the entire pipeline into the discarded second return value before buildPipelineStatement performs its bounded render. The fresh evidence after the earlier pipeline-capacity fix is this new unbounded summary path: a 1 MiB SET is still copied, and large pipelines are fully formatted despite the default cap; derive the summary directly from command names instead.

Useful? React with 👍 / 👎.

return appendStringTruncated(b, placeholder, limit)
}

s := fmt.Sprint(v)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bound non-struct custom argument formatting

When a valid command argument implements encoding.BinaryMarshaler on a large map, array, or pointer to an array, it reaches this fallback because the bounded branches only cover strings, byte slices, and struct-backed values. The fresh evidence after the struct-backed custom-argument fix is that a type Blob [1<<20]byte with MarshalBinary still makes fmt.Sprint allocate a multi-megabyte decimal representation before the 512-byte truncation is applied; use a bounded placeholder for the remaining custom backing kinds.

Useful? React with 👍 / 👎.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread extra/redisotel/statement.go Resolved

@ndyakov ndyakov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ronitgupta138, and apologies for the wait on this round. The redacting-Stringer leak is closed (SET k Secret("hunter2") gives SET k ***), the limit lives in rediscmd now, and the title is fixed. Two things block:

  1. A String() that panics now escapes the tracing hook. Master formats the panic inside fmt, and the statement becomes %!v(PANIC=String method: ...). This PR panics before next runs, which breaks the hook contract in AGENTS.md. Recover around the String() call, and fall back to fmt.Sprint, which recovers by itself (inline).
  2. pipelineSummary still renders every argument with a limit of 0, and the string is then thrown away. This happens even when db.statement is off. A pipeline with one 1 MiB SET costs about 0.56 ms and 1 MB of allocation per pipeline here, against about 2 KB with the limit. The summary only needs the command names, so build it from Name() (inline).

Not blocking, but it is the same bug class: a BinaryMarshaler on a non-struct type, such as a [1<<20]byte array, still goes through fmt.Sprint. At a 512-byte limit that costs about 21 ms and 12.7 MB per command. The placeholder you added for structs can cover every kind that is not string or []byte. Either here or as a follow-up.

Comment thread extra/rediscmd/rediscmd.go Outdated
if v == nil || (reflect.ValueOf(v).Kind() == reflect.Pointer && reflect.ValueOf(v).IsNil()) {
return append(b, "<nil>"...)
}
return appendStringTruncated(b, s.String(), limit)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

String() runs here with no recover. A value whose String() panics now crashes the tracing hook, and the hook contract (AGENTS.md) says hooks must not panic. Master's fmt.Sprint recovered from it. Call through a helper:

func safeString(s fmt.Stringer) (out string) {
	defer func() {
		if recover() != nil {
			out = fmt.Sprint(s) // fmt recovers the panic itself
		}
	}()
	return s.String()
}

Then use appendStringTruncated(b, safeString(s), limit).

Comment thread extra/redisotel/statement.go Outdated
}

func pipelineSummary(cmds []redis.Cmder) string {
summary, _ := rediscmd.CmdsStringLimit(cmds, 0)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This renders every argument, including 1 MiB values, and discards the statement. It runs on every pipeline, even with db.statement off. The span name only needs the unique command names (up to 10). Add a rediscmd helper that returns those names and use it here, so the summary never renders arguments.

… from command names

- Recover around String() call in rediscmd using safeString helper and fall back
  to fmt.Sprint to contain Stringer panics within statement rendering.
- Add rediscmd.CmdsSummary to extract up to 10 unique command names directly
  without formatting or allocating command arguments for pipeline spans.
- Use type placeholder for non-string/non-[]byte custom BinaryMarshaler values
  (arrays, maps, structs) when truncation is enabled to avoid large memory allocations.
- Add unit tests verifying panicking Stringer recovery, CmdsSummary zero-arg
  formatting, and non-struct custom type truncation.

Signed-off-by: Ronit Gupta <[email protected]>
@ronitgupta138

Copy link
Copy Markdown
Author

Thanks @ndyakov for the thorough review. Addressed in commit 29a2863:

  1. Panic in String() contained: Added safeString in extra/rediscmd using a deferred recover() falling back to fmt.Sprint(s) (which recovers itself and formats %!v(PANIC=String method: ...)), preserving the tracing hook contract.
  2. Pipeline summary argument allocation eliminated: Added rediscmd.CmdsSummary to extract up to 10 unique command names directly from cmd.FullName() without rendering or allocating any command arguments, and wired pipelineSummary to use it.
  3. Bounded custom non-struct BinaryMarshaler values: Generalized the bounded type placeholder (<package.Type>) to cover all non-string/non-[]byte custom types when truncation is enabled (such as [1<<20]byte arrays and maps), avoiding multi-megabyte fmt.Sprint formatting.

Added unit tests in both rediscmd and redisotel covering panicking Stringer recovery, CmdsSummary argument-free name extraction, and custom array marshaler placeholder formatting.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 29a2863f85

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +479 to +480
if val.Kind() == reflect.String {
return appendStringTruncated(b, val.String(), limit)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve error-backed redaction before reflecting strings

When a valid argument is a named string that implements both error and encoding.BinaryMarshaler, the old fmt.Sprint path used its Error() representation, but this reflection branch now reads the backing string directly. For a secret wrapper whose Error() returns *** while MarshalBinary() returns hunter2, both default redisotel tracing and existing rediscmd.CmdString callers now export hunter2; honor panic-safe error formatting before exposing the underlying value.

Useful? React with 👍 / 👎.

func safeString(s fmt.Stringer) (out string) {
defer func() {
if recover() != nil {
out = fmt.Sprint(s) // fmt recovers the panic itself

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Guard the recovery fallback against nested formatter panics

When a valid encoding.BinaryMarshaler's String() panics with a value whose own String or Error formatter also panics, this recovery handler calls fmt.Sprint(s), which re-enters String() and Go's nested-panic guard re-panics while formatting the panic value; the panic therefore still escapes ProcessHook before next runs. Fresh evidence after the simple Stringer recovery is this nested formatter path; recover the fallback too or emit a fixed placeholder.

AGENTS.md reference: AGENTS.md:L117-L122

Useful? React with 👍 / 👎.

if v == nil || (reflect.ValueOf(v).Kind() == reflect.Pointer && reflect.ValueOf(v).IsNil()) {
return append(b, "<nil>"...)
}
return appendStringTruncated(b, safeString(s), limit)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid materializing full Stringer output before truncation

When truncation is enabled and a valid encoding.BinaryMarshaler also implements fmt.Stringer by converting a large []byte payload to a string, safeString(s) materializes the entire result before the 512-byte budget is applied. Fresh evidence after the bounded reflection fix is that this prioritized Stringer branch bypasses the bounded byte-slice path, so a 1 MiB argument still adds a payload-sized synchronous tracing allocation; use a bounded placeholder or path when String output cannot be produced incrementally.

Useful? React with 👍 / 👎.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit 29a2863. Configure here.

b = append(b, '\n')
}
b = AppendCmd(b, cmd)
b = AppendCmdLimit(b, cmd, maxLen)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pipeline summary drops later commands

Low Severity

CmdsStringLimit builds its command-name summary in the same loop that truncates the statement, and breaks once the byte budget is exhausted. Later unique names never enter the summary even though they are still in the pipeline. CmdsSummary already walks the full command list for this, which is why tracing uses it for span names, but the new limited API still returns the partial list.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 29a2863. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

otel: truncate values in span attributes

4 participants