Skip to content

test: vuln-hunt 2026-05-18 regression suite (blocked-attack & isolation tests)#257

Merged
AlexandreYang merged 27 commits into
mainfrom
alex/vuln-campain
May 19, 2026
Merged

test: vuln-hunt 2026-05-18 regression suite (blocked-attack & isolation tests)#257
AlexandreYang merged 27 commits into
mainfrom
alex/vuln-campain

Conversation

@AlexandreYang

@AlexandreYang AlexandreYang commented May 18, 2026

Copy link
Copy Markdown
Member

Summary

Adds regression test coverage from the vuln-hunt 2026-05-18-initial-audit campaign — 18 commits, 40 new files, ~1.6k lines of test scaffolding. All tests assert behavior that is already correct in the implementation; they exist to lock that behavior in and catch future regressions.

Areas covered

Builtins (Go tests under builtins/*/)

  • cut, du, pwd, sed, xargs — blocked-attack regressions

Builtin scenarios (tests/scenarios/cmd/)

  • uname-- end-of-flags marker and --help short-circuit

Interpreter (Go tests under interp/ and analysis/)

  • Signal handling vuln-hunt tests (analysis/ + interp/)
  • Redirect subsystem vuln-hunt tests
  • Readonly enforcement vuln-hunt tests
  • /dev/null redirect pentest
  • while clause vuln-hunt tests

Shell scenarios (tests/scenarios/shell/)

  • blocked_commands: nameref, typeset
  • case_clause: expansion non-reparsing
  • errors: cmd-not-found subshell exit code, unknown-cmd stderr redirect
  • field_splitting: IFS backslash handling
  • function: expansion non-reparsing
  • inline_var: 7 scenarios covering cmdsub/redirect/digit/multi-assign restoration semantics
  • readonly: cmd-subst and redirect interactions
  • redirections: failed-redirect state isolation, var-as-redirect-target blocked
  • var_expand: metacharacters-in-value not reparsed, cmdsub-in-cond blocked
  • while_clause: continue overflow/huge-N, redirect blocking, subshell var isolation, brace-cond with exit
  • builtins_and_features: trap via expansion

Test plan

  • go test ./... -timeout 120s
  • go test ./tests/ -run TestShellScenarios -timeout 120s
  • RSHELL_BASH_TEST=1 go test ./tests/ -run TestShellScenariosAgainstBash -timeout 120s (scenarios that intentionally diverge from bash are marked skip_assert_against_bash)

5 new scenarios + 3 Go tests for the while_clause shell feature:

- redirect_loop_body_write_blocked: write redirect on loop body rejected
- redirect_loop_body_input_outside_allowed_blocked: input redirect on loop
  body honors AllowedPaths
- cmdsub_forbidden_in_cond_blocked: command substitution in cond honors
  AllowedPaths
- subshell_loop_var_isolation: variables assigned in a subshell-wrapped
  while loop do not leak to the parent
- while_brace_cond_with_exit: `exit N` in a brace-grouped cond exits the
  shell with status N before body runs
- TestUntilFalseRespectsContextCancellation: symmetric twin of the
  existing while-true ctx-cancel test
- TestSubshellInfiniteWhileRespectsContextCancellation: outer ctx cancel
  propagates through subshell into inner infinite while
- TestWhileEmptyCondIsParserError: `while ; do :; done` rejected at parse
…ssions

7 new scenarios documenting inline-command-variable behaviour under hostile
inputs (redirects, denied cmdsub, disallowed commands, IFS scoping, multi-
assignment, invalid identifiers).

The associated FINDING for same-name multi-assignment leak is tracked
privately in the rshell-vuln-hunt repo until a fix lands here.
Adds 3 regression tests:
- -I substituting cmdName remains bound by CommandAllowed post-substitution
- infinite NUL stream in default mode honors the 30s context timeout
- -L value validation mirrors -n (0, negative, > HardMaxArgs rejected)
5 regression tests covering: symlink-out-of-sandbox rejected;
--output-delimiter forwards control bytes within 1MB cap;
1MiB+ line exits 1; -b range overflow at strconv rejected;
-b1-99999999999 processes correctly bounded by line length.
Adds 2 regression tests:
- heredoc post-expansion cap is reachable only as defense-in-depth;
  layered per-var (1 MiB) + per-runner storage (2 MiB) caps fire first
- heredoc writer goroutine honors ctx cancel between 32 KiB chunks
Adds 1 regression test pinning that a symlink whose target lies outside
AllowedPaths is observable in pwd output but reads through the symlink
remain blocked by the sandbox.
@AlexandreYang AlexandreYang changed the title test(uname): vuln-hunt regression tests for end-of-flags and --help test: vuln-hunt 2026-05-18 regression suite (blocked-attack & isolation tests) May 19, 2026
…ectations

The four scenarios were failing CI because they relied on shell features
rshell does not implement, or because their expectations did not match
either rshell or bash behavior:

- shell/while_clause/continue_huge_n: used `$((i+1))` arithmetic expansion
  (not supported) and expected `continue 999999` to break out of the loop.
  Bash actually clamps N to the nesting depth, so `continue 999999` in a
  one-deep loop is equivalent to `continue 1`. Rewritten with an empty/
  non-empty sentinel and corrected expectation.
- shell/field_splitting/ifs_backslash: used `set --` plus `$#`/`$1` which
  rshell does not support. Rewritten to drive a `for x in $s` loop, which
  exercises the same IFS field-splitting behavior.
- shell/case_clause/expansion_does_not_reparse: original script
  `$KW x in y) echo no;; esac` is a parse-time error before $KW ever
  expands. Reduced to `KW=case; $KW`, which still demonstrates that the
  expanded "case" word is treated as a command name, not a keyword.
- shell/function/expansion_does_not_reparse: only the expected stderr
  substring was wrong — rshell emits "unknown command", not bash's
  "command not found". Updated stderr_contains accordingly.
@AlexandreYang AlexandreYang added the verified/analysis Human-reviewed static analysis changes label May 19, 2026
Audited the 27 new scenarios in this PR that carry
skip_assert_against_bash: true and verified each script against bash in
debian:bookworm-slim. For 6 of them, bash actually produces output that
matches the scenario's expect block, so the skip flag was unnecessary
and was hiding free coverage:

- cmd/uname/errors/vuln_hunt_help_with_garbage.yaml
- shell/errors/cmd_not_found_in_subshell_exit_code.yaml
- shell/errors/unknown_cmd_stderr_redirect.yaml
- shell/heredoc_dash/expansion_keeps_tabs.yaml
- shell/var_expand/basic/non_ascii_identifier_splits_at_boundary.yaml
- shell/while_clause/continue_overflow.yaml

The remaining 21 keep the flag for legitimate reasons (sandbox
restrictions, blocked builtins, or rshell-specific error wording that
does not match bash's "command not found" / "syntax error").
On Windows, the error message contains backslash path separators
(`openat etc\passwd: ...`) and does not match a stderr_contains check
for "/etc/passwd". Match on "passwd" instead, which is sufficient to
distinguish the failed read from any other stderr output and works
on all platforms.
@AlexandreYang
AlexandreYang marked this pull request as ready for review May 19, 2026 09:56
@AlexandreYang
AlexandreYang requested a review from thieman as a code owner May 19, 2026 09:56

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 89c398b4d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread interp/tests/redir_devnull_pentest_test.go Outdated
Codex flagged that TestPentestRedirNoFileCreated built its check script
as "ls "+dir, which on Windows tokenizes backslashes in the temp path
as shell escapes — and because the test ignored stderr/exit code, any
ls failure would silently leave stdout empty and pass the assertion.

Replaced the shell ls with a direct os.ReadDir(dir) and assert.Empty()
to make the check both cross-platform and fail-loud.
@AlexandreYang
AlexandreYang added this pull request to the merge queue May 19, 2026
Merged via the queue into main with commit 89f7403 May 19, 2026
40 checks passed
@AlexandreYang
AlexandreYang deleted the alex/vuln-campain branch May 19, 2026 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

verified/analysis Human-reviewed static analysis changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants