fix: stop evaluating step backticks by default#2075
Conversation
📝 WalkthroughWalkthroughThe changes introduce field-specific eval-option hooks to ExecutorCapabilities (CommandEvalOptions, ScriptEvalOptions, ConfigEvalOptions) and new Step methods that compute eval options by defaulting to WithoutSubstitute() then applying executor-provided options. A new EvalStepString function enforces substitution-disabled evaluation for step fields, with the legacy EvalOptions method deprecated. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 7/8 reviews remaining, refill in 7 minutes and 30 seconds.Comment |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
internal/runtime/builtin/docker/executor.go (1)
471-484:⚠️ Potential issue | 🟠 Major | ⚡ Quick winUse step-aware evaluation for
container.envvalues too.
ct.Envstill flows throughevalEnvSequentially, and Line 521 there still usesruntime.EvalString(...). That leaves backtick substitution enabled for step-owned container env values, so inputs likeFOO=\uname`can still execute even though the other container fields were moved toEvalStepString`.Suggested fix
func evalEnvSequentially(ctx context.Context, envs []string) ([]string, error) { if len(envs) == 0 { return envs, nil } resolved := make(map[string]string, len(envs)) result := make([]string, 0, len(envs)) for _, entry := range envs { key, rawVal, found := strings.Cut(entry, "=") if !found { result = append(result, entry) continue } - val, err := runtime.EvalString(ctx, rawVal, eval.WithVariables(resolved)) + val, err := runtime.EvalStepString(ctx, rawVal, eval.WithVariables(resolved)) if err != nil { return nil, fmt.Errorf("env %s: %w", key, err) } resolved[key] = val result = append(result, key+"="+val) } return result, nil }Also applies to: 490-498
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/runtime/builtin/docker/executor.go` around lines 471 - 484, ct.Env is still evaluated via evalEnvSequentially which uses runtime.EvalString and thus allows backtick substitution for step-owned env values; change the env evaluation to the step-aware evaluator used for other container fields (i.e., use EvalStepString instead of runtime.EvalString). Update evalEnvSequentially (or create a new evalEnvStepSequentially) to call runtime.EvalStepString for each env value so container environment variables are evaluated with step scope/escaping consistent with ct.Volumes, ct.Ports, ct.Command and ct.Shell.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@internal/runtime/builtin/docker/executor.go`:
- Around line 471-484: ct.Env is still evaluated via evalEnvSequentially which
uses runtime.EvalString and thus allows backtick substitution for step-owned env
values; change the env evaluation to the step-aware evaluator used for other
container fields (i.e., use EvalStepString instead of runtime.EvalString).
Update evalEnvSequentially (or create a new evalEnvStepSequentially) to call
runtime.EvalStepString for each env value so container environment variables are
evaluated with step scope/escaping consistent with ct.Volumes, ct.Ports,
ct.Command and ct.Shell.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: efea73cf-4246-4ae1-b270-a485a17f99ff
📒 Files selected for processing (16)
internal/core/capabilities.gointernal/core/capabilities_test.gointernal/runtime/builtin/agentstep/executor.gointernal/runtime/builtin/chat/executor.gointernal/runtime/builtin/command/command.gointernal/runtime/builtin/command/eval_options_test.gointernal/runtime/builtin/docker/eval_options_test.gointernal/runtime/builtin/docker/executor.gointernal/runtime/builtin/jq/jq.gointernal/runtime/builtin/ssh/ssh.gointernal/runtime/builtin/ssh/ssh_test.gointernal/runtime/builtin/template/template.gointernal/runtime/data.gointernal/runtime/eval.gointernal/runtime/node.gointernal/runtime/node_internal_test.go
Summary
Stop evaluating backtick command substitution in step-owned fields by default.
What Changed
Why
Step-owned text was being routed through the generic evaluator pipeline, which treats backticks as command substitution. That made prompts, templates, config strings, and other non-shell step fields brittle: a rendered value containing fenced Markdown or YAML could be executed accidentally during step setup instead of being passed through literally.
Validation
go test ./internal/core ./internal/runtime ./internal/runtime/builtin/command ./internal/runtime/builtin/docker ./internal/runtime/builtin/ssh -run 'Test(Step_EvalOptions|EvalExecutorConfig_DefaultPreservesLiteralCodeFencesInData|SetupExecutor_Harness(Command|Script)PreservesLiteralCodeFences|CommandExecutor_GetEvalOptions|DockerExecutor_GetEvalOptions|SSHExecutor_GetEvalOptions)' -count=1go test ./internal/core/... ./internal/runtime/... -count=1go test ./... -run ^ -count=1Summary by CodeRabbit
Refactor
Bug Fixes