test: vuln-hunt 2026-05-18 regression suite (blocked-attack & isolation tests)#257
Merged
Conversation
5 new scenarios + 3 Go tests for the while_clause shell feature: - redirect_loop_body_write_blocked: write redirect on loop body rejected - redirect_loop_body_input_outside_allowed_blocked: input redirect on loop body honors AllowedPaths - cmdsub_forbidden_in_cond_blocked: command substitution in cond honors AllowedPaths - subshell_loop_var_isolation: variables assigned in a subshell-wrapped while loop do not leak to the parent - while_brace_cond_with_exit: `exit N` in a brace-grouped cond exits the shell with status N before body runs - TestUntilFalseRespectsContextCancellation: symmetric twin of the existing while-true ctx-cancel test - TestSubshellInfiniteWhileRespectsContextCancellation: outer ctx cancel propagates through subshell into inner infinite while - TestWhileEmptyCondIsParserError: `while ; do :; done` rejected at parse
…ssions 7 new scenarios documenting inline-command-variable behaviour under hostile inputs (redirects, denied cmdsub, disallowed commands, IFS scoping, multi- assignment, invalid identifiers). The associated FINDING for same-name multi-assignment leak is tracked privately in the rshell-vuln-hunt repo until a fix lands here.
Adds 3 regression tests: - -I substituting cmdName remains bound by CommandAllowed post-substitution - infinite NUL stream in default mode honors the 30s context timeout - -L value validation mirrors -n (0, negative, > HardMaxArgs rejected)
5 regression tests covering: symlink-out-of-sandbox rejected; --output-delimiter forwards control bytes within 1MB cap; 1MiB+ line exits 1; -b range overflow at strconv rejected; -b1-99999999999 processes correctly bounded by line length.
Adds 2 regression tests: - heredoc post-expansion cap is reachable only as defense-in-depth; layered per-var (1 MiB) + per-runner storage (2 MiB) caps fire first - heredoc writer goroutine honors ctx cancel between 32 KiB chunks
Adds 1 regression test pinning that a symlink whose target lies outside AllowedPaths is observable in pwd output but reads through the symlink remain blocked by the sandbox.
…isolation scenario
…tripped + empty-post-strip-lines
…directory + --compress-program rejection
…ectations The four scenarios were failing CI because they relied on shell features rshell does not implement, or because their expectations did not match either rshell or bash behavior: - shell/while_clause/continue_huge_n: used `$((i+1))` arithmetic expansion (not supported) and expected `continue 999999` to break out of the loop. Bash actually clamps N to the nesting depth, so `continue 999999` in a one-deep loop is equivalent to `continue 1`. Rewritten with an empty/ non-empty sentinel and corrected expectation. - shell/field_splitting/ifs_backslash: used `set --` plus `$#`/`$1` which rshell does not support. Rewritten to drive a `for x in $s` loop, which exercises the same IFS field-splitting behavior. - shell/case_clause/expansion_does_not_reparse: original script `$KW x in y) echo no;; esac` is a parse-time error before $KW ever expands. Reduced to `KW=case; $KW`, which still demonstrates that the expanded "case" word is treated as a command name, not a keyword. - shell/function/expansion_does_not_reparse: only the expected stderr substring was wrong — rshell emits "unknown command", not bash's "command not found". Updated stderr_contains accordingly.
Audited the 27 new scenarios in this PR that carry skip_assert_against_bash: true and verified each script against bash in debian:bookworm-slim. For 6 of them, bash actually produces output that matches the scenario's expect block, so the skip flag was unnecessary and was hiding free coverage: - cmd/uname/errors/vuln_hunt_help_with_garbage.yaml - shell/errors/cmd_not_found_in_subshell_exit_code.yaml - shell/errors/unknown_cmd_stderr_redirect.yaml - shell/heredoc_dash/expansion_keeps_tabs.yaml - shell/var_expand/basic/non_ascii_identifier_splits_at_boundary.yaml - shell/while_clause/continue_overflow.yaml The remaining 21 keep the flag for legitimate reasons (sandbox restrictions, blocked builtins, or rshell-specific error wording that does not match bash's "command not found" / "syntax error").
On Windows, the error message contains backslash path separators (`openat etc\passwd: ...`) and does not match a stderr_contains check for "/etc/passwd". Match on "passwd" instead, which is sufficient to distinguish the failed read from any other stderr output and works on all platforms.
AlexandreYang
marked this pull request as ready for review
May 19, 2026 09:56
AlexandreYang
requested review from
astuyve,
julesmcrt,
matt-dz and
val06
as code owners
May 19, 2026 09:56
julesmcrt
approved these changes
May 19, 2026
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 89c398b4d9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex flagged that TestPentestRedirNoFileCreated built its check script as "ls "+dir, which on Windows tokenizes backslashes in the temp path as shell escapes — and because the test ignored stderr/exit code, any ls failure would silently leave stdout empty and pass the assertion. Replaced the shell ls with a direct os.ReadDir(dir) and assert.Empty() to make the check both cross-platform and fail-loud.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds regression test coverage from the
vuln-hunt 2026-05-18-initial-auditcampaign — 18 commits, 40 new files, ~1.6k lines of test scaffolding. All tests assert behavior that is already correct in the implementation; they exist to lock that behavior in and catch future regressions.Areas covered
Builtins (Go tests under
builtins/*/)cut,du,pwd,sed,xargs— blocked-attack regressionsBuiltin scenarios (
tests/scenarios/cmd/)uname—--end-of-flags marker and--helpshort-circuitInterpreter (Go tests under
interp/andanalysis/)analysis/+interp/)/dev/nullredirect pentestwhileclause vuln-hunt testsShell scenarios (
tests/scenarios/shell/)blocked_commands:nameref,typesetcase_clause: expansion non-reparsingerrors: cmd-not-found subshell exit code, unknown-cmd stderr redirectfield_splitting: IFS backslash handlingfunction: expansion non-reparsinginline_var: 7 scenarios covering cmdsub/redirect/digit/multi-assign restoration semanticsreadonly: cmd-subst and redirect interactionsredirections: failed-redirect state isolation, var-as-redirect-target blockedvar_expand: metacharacters-in-value not reparsed, cmdsub-in-cond blockedwhile_clause: continue overflow/huge-N, redirect blocking, subshell var isolation, brace-cond with exitbuiltins_and_features: trap via expansionTest plan
go test ./... -timeout 120sgo test ./tests/ -run TestShellScenarios -timeout 120sRSHELL_BASH_TEST=1 go test ./tests/ -run TestShellScenariosAgainstBash -timeout 120s(scenarios that intentionally diverge from bash are markedskip_assert_against_bash)