Skip to content

Commit 6dfe9da

Browse files
luisleo526claude
andcommitted
tests, docs: REV's follow-ups on the scratch series: the sink_fd rows name their real subject and cause, the capability row's cells are meaningful and disclosed, the predicate scan covers the signed spellings, the docs state the ENOTSUP exception
tests/test_run_json_sink_fd.py: the comment of the unjailed TMPDIR row now says the pre-series failure came from the native writer opening work/bad.sink in the mode-0555 work directory (sink_io), not from TMPDIR; new row test_native_fd_fsize_zero_path_sink_refused_in_read_only_work restores that subject (work 0555, TMPDIR 0755, report_sink_failed sink_io, nothing created); the TMPDIR-cause row runs without an arena, without PINEFORGE_COMMAND_SCRATCH_DIR and PINEFORGE_RUN_JSON_PROFILE, and proves DAC enforcement with an os.open probe first (visible pytest.skip with its reason otherwise). tests/test_default_scratch_effective_credentials.cpp nonroot_row() only (row() and main() unchanged): the setreuid outcome check can fail (EPERM or EINVAL accepted, credentials verified on both branches), the disclosure text is a pure fixture function pinned in test_scratch_capability_parse (every omitted cell "<cell> omitted (<reason>)", "missing, unreadable or malformed"), four more parser cases (0x80, -80, +80, space after the colon). tests/test_scratch_filesystem_predicate.py: the scan and its self-test also cover -2054924042(L) and 0xffffffff858458f6. docs/design/o1-report-contract.md and docs/pages/run-failure-codes.md: one paragraph each states the ENOTSUP exception (no probe, or a real errno 95 from stat/faccessat/statfs, keeps the default candidate as in 1.6.3 with no tmpfs claim and no note; /var/tmp selected without the rejected-candidate note when its probe answers ENOTSUP) and qualifies, with "except as described below", the sentences "A failed filesystem query stops default resolution with its syscall errno and names the failed candidate at every create wrapper; it never permits fallback." and "If creation in the selected `/var/tmp` fails, the detail names that directory, the rejected first candidate and `set TMPDIR to a disk-backed directory`." (o1-report-contract.md) and "A failed query stops resolution with its syscall errno and names the failed candidate." and "failures and all-memory refusals" (run-failure-codes.md); the other sentences keep their words and only their line wrapping changed (same number of lines in each block). No production code changed. Mutations m15 (setreuid EFAULT) and m16 (signed magic). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
1 parent 69b0b6f commit 6dfe9da

7 files changed

Lines changed: 165 additions & 60 deletions

‎docs/design/o1-report-contract.md‎

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -124,20 +124,20 @@ but no implementation must not be reported as supporting any new mode.
124124
pages, formatting scratch, indexes and charged I/O windows may not.
125125
Completion requires a validated complete sink receipt.
126126

127-
File-backed scratch for ordinary runs requires disk backing even when no sink
128-
or scratch directory was configured. The default selector first checks that the
129-
candidate is a searchable directory, then checks the filesystem type with `statfs`:
130-
use nonempty `TMPDIR` if disk-backed; when `TMPDIR` is unset
131-
or empty, retain disk-backed `/tmp` as the legacy first candidate. If that
132-
candidate is proven tmpfs/ramfs, try `/var/tmp`, also only if disk-backed. A failed
133-
filesystem query stops default resolution with its syscall errno and names the
134-
failed candidate at every create wrapper; it never permits fallback. Directory
135-
names never determine the medium, and tmpfs/ramfs is never silently selected.
136-
Once a disk-backed candidate is selected, ordinary creation/write failures keep
137-
their existing failure kinds and errno. Every create wrapper retains its established message prefix and
138-
appends the directory-selection detail. If creation in the selected `/var/tmp`
139-
fails, the detail names that directory, the rejected first candidate and
140-
`set TMPDIR to a disk-backed directory`.
127+
File-backed scratch for ordinary runs requires disk backing even when no sink or scratch directory was configured.
128+
The default selector first checks that the candidate is a searchable directory, then checks the filesystem type with `statfs`:
129+
use nonempty `TMPDIR` if disk-backed; when `TMPDIR` is unset or empty, retain disk-backed `/tmp` as the legacy first candidate.
130+
If that candidate is proven tmpfs/ramfs, try `/var/tmp`, also only if disk-backed. A failed filesystem query stops default resolution
131+
with its syscall errno and names the failed candidate at every create wrapper; it never permits fallback, except as described below.
132+
Directory names never determine the medium, and tmpfs/ramfs is never silently selected. Once a disk-backed candidate is selected,
133+
ordinary creation/write failures keep their existing failure kinds and errno. Every create wrapper retains its established message prefix
134+
and appends the directory-selection detail. If creation in the selected `/var/tmp` fails, the detail names that directory, the rejected
135+
first candidate and `set TMPDIR to a disk-backed directory`, except as described below.
136+
137+
An ENOTSUP probe gives no filesystem evidence: use the default candidate as in 1.6.3 (error 0, no tmpfs claim, no refusal note).
138+
This covers hosts with no probe (neither Linux nor macOS) and real ENOTSUP from stat/faccessat/statfs; the test is by errno value,
139+
so on Linux ENOTSUP and EOPNOTSUPP are both 95. A FUSE/9p/network mount answering 95 keeps its candidate; a later create failure
140+
keeps the plain 1.6.3 text. If the first candidate is proven tmpfs/ramfs and /var/tmp answers ENOTSUP, select /var/tmp without the rejected-candidate note.
141141

142142
If both default candidates are proven tmpfs/ramfs, refuse with `ENODEV` and
143143
`scratch requires disk backing`. The failure detail names every candidate tried

‎docs/pages/run-failure-codes.md‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -242,16 +242,16 @@ per-store arena share carries this same typed verdict, with needed bytes
242242
computed as the arena size satisfying that store's required extent. A separate
243243
command byte-budget refusal retains its status-only I/O behavior.
244244

245-
File-backed scratch must have disk backing. With no scratch directory configured,
246-
the engine checks filesystem type with `statfs`, using nonempty `TMPDIR` if
247-
disk-backed, or the legacy `/tmp` candidate when `TMPDIR` is unset or empty.
248-
Only a successful query proving that first candidate is tmpfs/ramfs permits
249-
trying `/var/tmp`, also checked by filesystem type. A failed query stops
250-
resolution with its syscall errno and names the failed candidate. Directory
251-
names are not a medium test; tmpfs/ramfs is never silently chosen. If both default
252-
candidates are proven tmpfs/ramfs, the engine retains `ENODEV` /
253-
`scratch requires disk backing`, names every candidate tried and its rejection
254-
reason in the failure detail, and adds `set TMPDIR to a disk-backed directory`.
245+
File-backed scratch must have disk backing. With no scratch directory configured, the engine checks filesystem type with `statfs`,
246+
using nonempty `TMPDIR` if disk-backed, or the legacy `/tmp` candidate when `TMPDIR` is unset or empty. Only a successful query proving
247+
that first candidate is tmpfs/ramfs permits trying `/var/tmp`, also checked by filesystem type. A failed query stops resolution with its
248+
syscall errno and names the failed candidate, except as described below. Directory names are not a medium test; tmpfs/ramfs is never silently chosen.
249+
If both default candidates are proven tmpfs/ramfs, the engine retains `ENODEV` / `scratch requires disk backing`, names every candidate tried and its rejection reason in the failure detail, and adds `set TMPDIR to a disk-backed directory`.
250+
251+
An ENOTSUP probe gives no filesystem evidence: use the default candidate as in 1.6.3 (error 0, no tmpfs claim, no refusal note).
252+
This covers hosts with no probe (neither Linux nor macOS) and real ENOTSUP from stat/faccessat/statfs; the test is by errno value,
253+
so on Linux ENOTSUP and EOPNOTSUPP are both 95. A FUSE/9p/network mount answering 95 keeps its candidate; a later create failure
254+
keeps the plain 1.6.3 text. If the first candidate is proven tmpfs/ramfs and /var/tmp answers ENOTSUP, select /var/tmp without the rejected-candidate note.
255255

256256
An explicit scratch/spill directory is kept as supplied and never falls back;
257257
explicit tmpfs/ramfs directories are now refused, including command scratch.
@@ -260,7 +260,7 @@ The explicit sink-path refusal
260260
still requires a sealed arena and never creates a backing file or falls back to
261261
`TMPDIR`, `/tmp` or `/var/tmp`. File-backed command scratch uses
262262
`scratch_io_failed` (class `resource`), including default-directory query
263-
failures and all-memory refusals. Native runs create command scratch at begin,
263+
failures other than ENOTSUP and all-memory refusals. Native runs create command scratch at begin,
264264
before strategy callbacks. Report scratch uses `report_sink_failed` with
265265
`reason: scratch_io`; auxiliary tables use `scratch_io_failed`; sink-path I/O
266266
uses `report_sink_failed` with `reason: sink_io`.

‎tests/scratch_capability_fixture.hpp‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,30 @@ inline ScratchEffectiveCapabilities parse_scratch_effective_capabilities(std::st
3636
(mask & (UINT64_C(1) << 2)) != 0}; // CAP_DAC_READ_SEARCH
3737
}
3838

39+
inline std::string describe_cells(const ScratchEffectiveCapabilities& capabilities) {
40+
std::string names;
41+
if (!capabilities.known) {
42+
names = "unknown (/proc/self/status CapEff missing, unreadable or malformed; "
43+
"capabilities may be present)";
44+
} else {
45+
if (capabilities.setuid) names += "CAP_SETUID(7) ";
46+
if (capabilities.dac_override) names += "CAP_DAC_OVERRIDE(1) ";
47+
if (capabilities.dac_read_search) names += "CAP_DAC_READ_SEARCH(2) ";
48+
if (names.empty()) names = "none relevant";
49+
else names.pop_back();
50+
}
51+
std::string description = "effective capabilities: " + names + "; cells: ";
52+
description += capabilities.may_setuid()
53+
? "setreuid EPERM omitted (CAP_SETUID may be present), "
54+
"child setreuid outcome and matching-credentials control"
55+
: "setreuid EPERM";
56+
description += ", owned-0700 TMPDIR positive control, ";
57+
description += capabilities.may_bypass_dac()
58+
? "mode-0000 EACCES omitted (DAC capabilities may be present)"
59+
: "mode-0000 EACCES";
60+
return description;
61+
}
62+
3963
inline ScratchEffectiveCapabilities read_scratch_effective_capabilities() {
4064
// No caller-supplied paths: only this fixed procfs file can be opened.
4165
std::puts("S4E capability inspection path: /proc/self/status (fixed path only)");

‎tests/test_default_scratch_effective_credentials.cpp‎

Lines changed: 8 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -10,24 +10,7 @@ void nonroot_row() {
1010
require(::geteuid() != 0 && ::getuid() == ::geteuid(),
1111
"non-root controls require matching real and effective uids");
1212
const auto capabilities = read_scratch_effective_capabilities();
13-
std::string capability_names;
14-
if (!capabilities.known) {
15-
capability_names = "unknown (/proc/self/status CapEff unreadable or malformed; "
16-
"capabilities may be present)";
17-
} else {
18-
if (capabilities.setuid) capability_names += "CAP_SETUID(7) ";
19-
if (capabilities.dac_override) capability_names += "CAP_DAC_OVERRIDE(1) ";
20-
if (capabilities.dac_read_search) capability_names += "CAP_DAC_READ_SEARCH(2) ";
21-
if (capability_names.empty()) capability_names = "none relevant";
22-
else capability_names.pop_back();
23-
}
24-
std::printf("effective capabilities: %s; cells: %s, owned-0700 TMPDIR positive control, %s\n",
25-
capability_names.c_str(),
26-
capabilities.may_setuid() ? "child setreuid outcome and matching-credentials control"
27-
: "setreuid EPERM",
28-
capabilities.may_bypass_dac()
29-
? "mode-0000 EACCES cell omitted (DAC capabilities may be present)"
30-
: "mode-0000 EACCES");
13+
std::puts(describe_cells(capabilities).c_str());
3114
if (!capabilities.may_setuid()) {
3215
scratch_child("non-root effective-credentials refusal", [] {
3316
errno = 0;
@@ -74,15 +57,19 @@ void nonroot_row() {
7457
errno = 0;
7558
const int result = ::setreuid(65534, 0);
7659
const int observed_error = errno;
77-
require(result == 0 || (result == -1 && observed_error != 0),
78-
"capability-aware setreuid must succeed or report a real errno");
60+
// EPERM: seccomp, LSM or no_new_privs; EINVAL: an unmapped namespace uid.
61+
require(result == 0 || observed_error == EPERM || observed_error == EINVAL,
62+
"capability-aware setreuid must succeed or report EPERM or EINVAL");
7963
if (result == 0) {
80-
require(::getuid() == 65534 && ::geteuid() == 0,
64+
require(::getuid() == 65534 && ::geteuid() == 0
65+
&& (::getuid() != original_real || ::geteuid() != original_effective),
8166
"successful setreuid must install the requested credentials");
8267
require(::setreuid(original_real, original_effective) == 0,
8368
"cannot restore child credentials after successful setreuid");
8469
std::puts("non-root setreuid(65534, 0): returned 0; child credentials restored");
8570
} else {
71+
require(::getuid() == original_real && ::geteuid() == original_effective,
72+
"failed setreuid must leave child real and effective uids unchanged");
8673
std::printf("non-root setreuid(65534, 0): returned -1 (errno %d)\n",
8774
observed_error);
8875
}

‎tests/test_run_json_sink_fd.py‎

Lines changed: 74 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -763,7 +763,8 @@ def sealed_arena_fd(size):
763763

764764
def child_run(source, extra, *, transport="pipe", stub=True, jail=False,
765765
work=None, temporary=None, fault=None, break_after=None, pnls=PNLS,
766-
without_arena=False, arena_path=None, arena_fd=None):
766+
without_arena=False, arena_path=None, arena_fd=None,
767+
without_scratch_overrides=False):
767768
event_read, event_write = os.pipe()
768769
with contextlib.ExitStack() as inherited, Stream(transport, break_after=break_after) as stream:
769770
command = [sys.executable, "-B", str(Path(__file__).resolve()),
@@ -786,8 +787,8 @@ def child_run(source, extra, *, transport="pipe", stub=True, jail=False,
786787
command += ["--jail"]
787788
command += ["--", "--ohlcv", str(source), *map(str, extra)]
788789
if jail and temporary is not None and "--profile" not in extra:
789-
# Match the wrapper's explicit, read-only scratch directory for
790-
# the path-sink refusal too; tempfile's directory probing is not
790+
# Match the wrapper's explicit scratch directory for the
791+
# path-sink refusal too; tempfile's directory probing is not
791792
# the native writer/open failure that this control exercises.
792793
command += ["--spill-dir", str(temporary)]
793794
if "--report-sink" not in extra:
@@ -797,6 +798,9 @@ def child_run(source, extra, *, transport="pipe", stub=True, jail=False,
797798
for name in ("PF_SCRATCH_ARENA_FD", "PF_SCRATCH_ARENA_BYTES",
798799
"PF_SCRATCH_WINDOW_BYTES"):
799800
environment.pop(name, None)
801+
if without_scratch_overrides:
802+
for name in ("PINEFORGE_COMMAND_SCRATCH_DIR", "PINEFORGE_RUN_JSON_PROFILE"):
803+
environment.pop(name, None)
800804
if temporary is not None:
801805
environment.update(TMPDIR=str(temporary), TMP=str(temporary), TEMP=str(temporary))
802806
try:
@@ -904,23 +908,81 @@ def test_native_fd_fsize_zero_without_arena_refuses_before_file_operations(tmp_p
904908
assert failure["reason"] == failure["args"]["reason"]
905909
assert len(actual[1].splitlines()) == 1 and "summary" not in failure
906910
assert actual[3] == b"" and snapshot(work, temporary) == before
911+
finally:
912+
work.chmod(0o755)
913+
temporary.chmod(0o755)
914+
915+
916+
def test_native_fd_fsize_zero_without_arena_tmpdir_cause(tmp_path):
917+
source = bars.csv_file(tmp_path, count=17)
918+
extra = ["--so", native_module(False), "--input-tf", "1", "--script-tf", "1"]
919+
work, temporary = tmp_path / "work", tmp_path / "tmp"
920+
work.mkdir()
921+
temporary.mkdir()
922+
work.chmod(0o555)
923+
temporary.chmod(0o555)
924+
try:
925+
try:
926+
probe = os.open(temporary / "probe", os.O_CREAT | os.O_EXCL | os.O_WRONLY)
927+
except PermissionError:
928+
pass
929+
else:
930+
os.close(probe)
931+
pytest.skip("mode-0555 TMPDIR permits create: root or DAC capabilities "
932+
"invalidate this Python row's permission precondition")
933+
before = snapshot(work, temporary)
907934
refused = child_run(source, [*extra, "--report-sink", str(work / "bad.sink")],
908-
stub=False, jail=True, work=work, temporary=temporary)
935+
stub=False, jail=True, work=work, temporary=temporary,
936+
without_arena=True, without_scratch_overrides=True)
909937
assert refused[0] == 1 and refused[2] == "", refused
910-
# This second run is the default (unjailed) profile with no arena, so the
911-
# command scratch is a directory file. L6's D1(ii) ruling: the command
912-
# scratch resolves TMPDIR like every other scratch and refuses at begin
913-
# (docs/pages/run-failure-codes.md, the default scratch directory policy:
914-
# scratch_io_failed, class resource). Before the series it used the literal
915-
# /tmp, so the read-only TMPDIR was first met by the sink ledger
916-
# (report_sink_failed). Nothing may be created in work or TMPDIR either way.
917-
assert json.loads(refused[1])["code"] == "scratch_io_failed"
938+
# With the unjailed profile and no arena or explicit scratch override,
939+
# command scratch resolves TMPDIR and its create fails before sink open.
940+
failure = json.loads(refused[1])
941+
assert failure["code"] == "scratch_io_failed"
942+
assert "command scratch create" in failure["error"], failure
943+
assert "requires disk backing" not in failure["error"], failure
918944
assert snapshot(work, temporary) == before
919945
finally:
920946
work.chmod(0o755)
921947
temporary.chmod(0o755)
922948

923949

950+
def test_native_fd_fsize_zero_path_sink_refused_in_read_only_work(tmp_path):
951+
source = bars.csv_file(tmp_path, count=17)
952+
extra = ["--so", native_module(False), "--input-tf", "1", "--script-tf", "1"]
953+
work, temporary = tmp_path / "work", tmp_path / "tmp"
954+
work.mkdir()
955+
temporary.mkdir()
956+
work.chmod(0o555)
957+
temporary.chmod(0o755)
958+
try:
959+
try:
960+
probe = os.open(work / "probe", os.O_CREAT | os.O_EXCL | os.O_WRONLY)
961+
except PermissionError:
962+
pass
963+
else:
964+
os.close(probe)
965+
pytest.skip("mode-0555 work permits create: root or DAC capabilities "
966+
"invalidate this Python row's permission precondition")
967+
before = snapshot(work)
968+
# At 3b76e98b86 command scratch used /tmp and succeeded. The old red was
969+
# the native writer opening work/bad.sink in mode-0555 work:
970+
# engine_report.cpp:1194-1198 -> report_sink_writer.cpp:491-493,
971+
# EACCES -> report_sink_failed {reason: sink_io}, before TMPDIR use.
972+
# Writable, disk-backed temporary restores that sink-open subject here.
973+
refused = child_run(source, [*extra, "--report-sink", str(work / "bad.sink")],
974+
stub=False, jail=True, work=work, temporary=temporary,
975+
without_arena=True, without_scratch_overrides=True)
976+
assert refused[0] == 1 and refused[2] == "", refused
977+
failure = json.loads(refused[1])
978+
assert failure["code"] == "report_sink_failed"
979+
assert failure["args"] == {"reason": "sink_io"}
980+
assert snapshot(work) == before
981+
finally:
982+
work.chmod(0o755)
983+
temporary.chmod(0o755)
984+
985+
924986
def test_native_jailed_fd_without_arena_refuses_before_scratch_or_sink(tmp_path):
925987
source = bars.csv_file(tmp_path, count=17)
926988
actual = child_run(source, ["--so", native_module(False), "--profile", "jailed"],

‎tests/test_scratch_capability_parse.cpp‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,13 @@ void expect(std::string_view line, bool known, bool setuid,
1212
throw std::runtime_error("wrong capability parse for '" + std::string(line) + "'");
1313
}
1414

15+
void expect_cells(const s4e_tmp::ScratchEffectiveCapabilities& capabilities,
16+
std::string_view expected) {
17+
const auto actual = s4e_tmp::describe_cells(capabilities);
18+
if (actual != expected)
19+
throw std::runtime_error("wrong capability cell disclosure: " + actual);
20+
}
21+
1522
void row() {
1623
expect("CapEff:\t0000000000000000", true, false, false, false);
1724
expect("CapEff:\t0000000000000080", true, true, false, false);
@@ -23,6 +30,10 @@ void row() {
2330
expect("CapEff:\t", false, false, false, false);
2431
expect("CapEff:\t0000000000000080junk", false, false, false, false);
2532
expect("CapEff:\t10000000000000000", false, false, false, false);
33+
expect("CapEff:\t0x80", false, false, false, false);
34+
expect("CapEff:\t-80", false, false, false, false);
35+
expect("CapEff:\t+80", false, false, false, false);
36+
expect("CapEff: 0000000000000080", true, true, false, false);
2637
expect("CapEff:\t0000000000000100", true, false, false, false);
2738
expect("CapEff:\t000001FFFFFFFFFF \t\r\n", true, true, true, true);
2839

@@ -32,6 +43,26 @@ void row() {
3243
if (none.may_setuid() || none.may_bypass_dac()
3344
|| !unknown.may_setuid() || !unknown.may_bypass_dac())
3445
throw std::runtime_error("unknown capabilities must conservatively permit both overrides");
46+
47+
expect_cells(none,
48+
"effective capabilities: none relevant; cells: setreuid EPERM, "
49+
"owned-0700 TMPDIR positive control, mode-0000 EACCES");
50+
expect_cells(s4e_tmp::parse_scratch_effective_capabilities("CapEff:\t0000000000000080"),
51+
"effective capabilities: CAP_SETUID(7); cells: "
52+
"setreuid EPERM omitted (CAP_SETUID may be present), "
53+
"child setreuid outcome and matching-credentials control, "
54+
"owned-0700 TMPDIR positive control, mode-0000 EACCES");
55+
expect_cells(s4e_tmp::parse_scratch_effective_capabilities("CapEff:\t0000000000000006"),
56+
"effective capabilities: CAP_DAC_OVERRIDE(1) CAP_DAC_READ_SEARCH(2); cells: "
57+
"setreuid EPERM, owned-0700 TMPDIR positive control, "
58+
"mode-0000 EACCES omitted (DAC capabilities may be present)");
59+
expect_cells(unknown,
60+
"effective capabilities: unknown (/proc/self/status CapEff missing, unreadable or malformed; "
61+
"capabilities may be present); cells: "
62+
"setreuid EPERM omitted (CAP_SETUID may be present), "
63+
"child setreuid outcome and matching-credentials control, "
64+
"owned-0700 TMPDIR positive control, "
65+
"mode-0000 EACCES omitted (DAC capabilities may be present)");
3566
}
3667
} // namespace
3768

0 commit comments

Comments
 (0)