Skip to content

Commit 434b98c

Browse files
authored
tests: attribute scratch-tmpfs watcher events to this process, not co-tenants (#521)
Removes a 1.8.0 cut-run risk: a co-tenant job on the same CI VM could turn these rows red. ## Problem `test_default_scratch_tmpfs_tmpdir` and `test_jailed_scratch_tmpfs_tmpdir` judge the host-shared `/dev/shm`, `/tmp` and `/var/tmp` by *any* file that appears there during the run (`pineforge-*` for the default row, any name for the jailed row). The watcher never identifies the creator, and `RUN_SERIAL` serialises only one ctest, not the second runner on the same 2-runner post VM. A co-tenant job can therefore turn a green engine run red (and the default row's `/var/tmp` positive claim could false-PASS on a co-tenant's file). ## Change (tests only) - The tmpfs candidate is a private `mkdtemp` directory under `/dev/shm` owned by the test (RAII cleanup confined to that directory) and is watched strictly. - In `/tmp` and `/var/tmp`, attribution is exact: the test executable interposes the engine's only temp-file creation path (`mkstemp`, `include/pineforge/detail/scratch_file.hpp:147`, `src/command_scratch_production.cpp:235`) **inside the test executable only**; no engine source, no `include/`, no build flag or option the shipped library sees (CMake changes link `dl`/`Threads` to the three test targets only, Linux only). - New `tests/test_scratch_watch_attribution.cpp` rows: a planted co-tenant file no longer fails the default or jailed row; a co-tenant file cannot satisfy the positive claim; files this process creates in `/tmp`, `/var/tmp` or `/dev/shm` still fail. Files touched: `tests/CMakeLists.txt`, `tests/default_scratch_tmpfs_fixture.hpp`, `tests/test_default_scratch_tmpfs_tmpdir.cpp`, `tests/test_jailed_scratch_tmpfs_tmpdir.cpp`, `tests/test_scratch_watch_attribution.cpp` (new). Route: CI-only; promotes no baseline. ## Limit For the shared roots `/tmp` and `/var/tmp` the negative claim sees only scratch created through `mkstemp` in this process (plus anything created inside the private TMPDIR). A future engine path that creates scratch there with another call (`open(O_CREAT)`, `O_TMPFILE`) would not be seen in those two roots. The positive `/var/tmp` claim and the private-directory watch are not affected. Today the only scratch creators are the two `mkstemp` sites above. ## Evidence - Linux build host, ctest of the touched targets plus the cmake lane-file checker tests, on the merged tree: job `rj-20261026t054615-1bd154` passed (the same selection passed on the earlier head in `rj-20261026t040636-1f56c7`: 15/15, including both scratch-tmpfs tests, the nine `test_scratch_watch_attribution*` cases, `test_pipe_writer_fd_flags` and `test_cr150_cmake_d` with its controls). - Red on the base, black-box (job `rj-20261026t050715-5b540a`): the UNCHANGED tests of the base commit run once without interference (green), then again while a separate process creates and removes `pineforge-*` files in `/dev/shm` and `/tmp` throughout the run; a judge that requires the green control and then exit 1 with the watcher's own assertion passed for both rows. This is the co-tenant failure this PR removes. - Debug and sanitizer profiles of this branch: job `rj-20261026t055037-a75341` passed on the merged tree (debug: 1179 tests, 100%; sanitizers: 1160 tests, 100%, including the scratch-attribution cases and both scratch-tmpfs rows built under the sanitizer interceptors). Main has since gained only unrelated checker-script files. - Not run on the author's machine. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
1 parent 4b41eae commit 434b98c

5 files changed

Lines changed: 656 additions & 7 deletions

‎tests/CMakeLists.txt‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7238,13 +7238,34 @@ endif()
72387238

72397239
add_test(NAME test_cr150_cmake_d COMMAND ${Python3_EXECUTABLE} -B ${PROJECT_SOURCE_DIR}/scripts/check_cr150_cmake_d.py)
72407240

7241+
# CR-352: regression rows retain the fixture API available on the red base.
7242+
if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
7243+
list(APPEND TEST_SOURCES test_scratch_watch_attribution)
7244+
pineforge_add_unit_test(test_scratch_watch_attribution)
7245+
target_link_libraries(test_scratch_watch_attribution PRIVATE ${CMAKE_DL_LIBS} Threads::Threads)
7246+
foreach(_pf_scratch_watch_case IN ITEMS
7247+
cotenant_default cotenant_jailed cotenant_positive
7248+
own_tmp own_transient_tmp own_other_tmp own_shm private_non_mkstemp header_scratch)
7249+
add_test(NAME test_scratch_watch_attribution_${_pf_scratch_watch_case}
7250+
COMMAND test_scratch_watch_attribution ${_pf_scratch_watch_case})
7251+
endforeach()
7252+
endif()
7253+
72417254
# CR-150: one file per lane in tests/cr150.d/<lane>.cmake (see tests/cr150.d/README.md).
72427255
file(GLOB CR150_LANE_CMAKE CONFIGURE_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/cr150.d/*.cmake")
72437256
list(SORT CR150_LANE_CMAKE)
72447257
foreach(cr150_lane_file IN LISTS CR150_LANE_CMAKE)
72457258
include("${cr150_lane_file}")
72467259
endforeach()
72477260

7261+
# --- CR-150 s4e-fix-tmp begin ---
7262+
# CR-352: only these opted-in existing targets need the interposer's linkage.
7263+
if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
7264+
target_link_libraries(test_default_scratch_tmpfs_tmpdir PRIVATE ${CMAKE_DL_LIBS} Threads::Threads)
7265+
target_link_libraries(test_jailed_scratch_tmpfs_tmpdir PRIVATE ${CMAKE_DL_LIBS} Threads::Threads)
7266+
endif()
7267+
# --- CR-150 s4e-fix-tmp end ---
7268+
72487269
# --- CR-150 core-cr150-l6b1 begin ---
72497270
# CR-150 L6B1: appended after every pre-existing line pin. A real source
72507271
# strategy module, with the same whole-archive link as release modules.

‎tests/default_scratch_tmpfs_fixture.hpp‎

Lines changed: 275 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
#pragma once
22

3-
#if defined(__linux__) && !defined(_GNU_SOURCE)
3+
#if defined(__linux__)
4+
#if !defined(_GNU_SOURCE)
45
#define _GNU_SOURCE
56
#endif
67

@@ -28,6 +29,12 @@
2829
#include <utility>
2930
#include <vector>
3031

32+
#if defined(PINEFORGE_TEST_SCRATCH_ATTRIBUTION)
33+
#include <dlfcn.h>
34+
#include <mutex>
35+
#include <string_view>
36+
#endif
37+
3138
namespace s4e_tmp {
3239

3340
inline void require(bool condition, const std::string& message) {
@@ -59,6 +66,127 @@ struct Environment {
5966
}
6067
};
6168

69+
#if defined(PINEFORGE_TEST_SCRATCH_ATTRIBUTION)
70+
inline bool under_directory(std::string_view path, std::string_view directory) {
71+
return path == directory || (path.size() > directory.size()
72+
&& path.compare(0, directory.size(), directory) == 0
73+
&& path[directory.size()] == '/');
74+
}
75+
76+
// Only names in the announced scratch roots may be inspected. Reject secret
77+
// directories and *.env by name even if they appear below an allowed root.
78+
inline bool scratch_inspection_path(std::string_view path) {
79+
if (!under_directory(path, "/dev/shm") && !under_directory(path, "/tmp")
80+
&& !under_directory(path, "/var/tmp")) return false;
81+
for (const auto root : {"/.config/zsh/keys", "/.config/pineforge"}) {
82+
const auto position = path.find(root);
83+
if (position != std::string_view::npos
84+
&& (position + std::strlen(root) == path.size()
85+
|| path[position + std::strlen(root)] == '/')) return false;
86+
}
87+
std::size_t start = 1;
88+
while (start < path.size()) {
89+
const auto end = path.find('/', start);
90+
const auto component = path.substr(start,
91+
end == std::string_view::npos ? path.size() - start : end - start);
92+
if (component == "." || component == ".." || component == ".ssh"
93+
|| component == ".aws" || component == ".codex"
94+
|| (component.size() >= 4 && component.substr(component.size() - 4) == ".env"))
95+
return false;
96+
if (end == std::string_view::npos) break;
97+
start = end + 1;
98+
}
99+
return true;
100+
}
101+
102+
class PrivateTmpdir {
103+
public:
104+
PrivateTmpdir() : path_("/dev/shm/pineforge-tmpwatch-XXXXXX") {
105+
std::printf("S4E owned TMPDIR template: %s (only this directory is cleaned)\n",
106+
path_.c_str());
107+
require(::mkdtemp(path_.data()) != nullptr, "cannot create the private tmpfs TMPDIR");
108+
}
109+
~PrivateTmpdir() {
110+
// No recursion or symlink following: remove only entries in the
111+
// directory this object created, then that directory itself.
112+
bool clean = true;
113+
const int opened = ::open(path_.c_str(), O_RDONLY | O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW);
114+
DIR* directory = opened >= 0 ? ::fdopendir(opened) : nullptr;
115+
if (!directory) {
116+
if (opened >= 0) ::close(opened);
117+
clean = false;
118+
} else {
119+
for (;;) {
120+
errno = 0;
121+
const auto* entry = ::readdir(directory);
122+
if (!entry) { if (errno) clean = false; break; }
123+
if (std::strcmp(entry->d_name, ".") == 0
124+
|| std::strcmp(entry->d_name, "..") == 0) continue;
125+
if (!scratch_inspection_path(path_ + "/" + entry->d_name)
126+
|| ::unlinkat(::dirfd(directory), entry->d_name, 0) != 0) clean = false;
127+
}
128+
if (::closedir(directory) != 0) clean = false;
129+
}
130+
if (::rmdir(path_.c_str()) != 0) clean = false;
131+
if (!clean) {
132+
std::fprintf(stderr, "cannot clean owned TMPDIR: %s\n", path_.c_str());
133+
std::abort();
134+
}
135+
}
136+
PrivateTmpdir(const PrivateTmpdir&) = delete;
137+
PrivateTmpdir& operator=(const PrivateTmpdir&) = delete;
138+
const char* path() const { return path_.c_str(); }
139+
private:
140+
std::string path_;
141+
};
142+
143+
struct MkstempTrace {
144+
const pid_t owner = ::getpid();
145+
std::vector<std::string> paths;
146+
bool failed = false;
147+
};
148+
149+
struct MkstempRegistry {
150+
std::mutex mutex;
151+
MkstempTrace* active = nullptr;
152+
};
153+
154+
inline MkstempRegistry& mkstemp_registry() {
155+
static MkstempRegistry registry;
156+
return registry;
157+
}
158+
159+
inline void fail_mkstemp_trace() {
160+
auto& registry = mkstemp_registry();
161+
std::lock_guard<std::mutex> lock(registry.mutex);
162+
if (registry.active && registry.active->owner == ::getpid()) registry.active->failed = true;
163+
}
164+
165+
inline void record_mkstemp(int descriptor, const char* path) noexcept {
166+
auto& registry = mkstemp_registry();
167+
std::lock_guard<std::mutex> lock(registry.mutex);
168+
auto* trace = registry.active;
169+
// A forked co-tenant inherits memory, never ownership of this witness.
170+
if (!trace || trace->owner != ::getpid()) return;
171+
try {
172+
require(scratch_inspection_path(path), "refused scratch inspection path by name");
173+
const auto link = std::string("/proc/self/fd/") + std::to_string(descriptor);
174+
std::array<char, 4096> target{};
175+
const auto length = ::readlink(link.c_str(), target.data(), target.size());
176+
require(length > 0 && static_cast<std::size_t>(length) < target.size(),
177+
"cannot read the returned mkstemp descriptor's name");
178+
const std::string observed(target.data(), static_cast<std::size_t>(length));
179+
require(observed == path || observed == std::string(path) + " (deleted)",
180+
"mkstemp path disagrees with its own descriptor");
181+
trace->paths.emplace_back(path); // Before the caller can unlink/close.
182+
} catch (...) {
183+
// Instrumentation must not throw across libc or turn a failed
184+
// observation into absence of scratch. Every query checks this bit.
185+
trace->failed = true;
186+
}
187+
}
188+
#endif // PINEFORGE_TEST_SCRATCH_ATTRIBUTION
189+
62190
// Every row owns its environment; inherited runner settings cannot turn the
63191
// default-directory witness into an arena or explicitly configured run.
64192
struct PlainEnvironment {
@@ -68,7 +196,12 @@ struct PlainEnvironment {
68196
Environment window{"PF_SCRATCH_WINDOW_BYTES", nullptr};
69197
Environment command_directory{"PINEFORGE_COMMAND_SCRATCH_DIR", nullptr};
70198
Environment command_cap{"PINEFORGE_COMMAND_SCRATCH_MAX_BYTES", nullptr};
199+
#if defined(PINEFORGE_TEST_SCRATCH_ATTRIBUTION)
200+
PrivateTmpdir directory;
201+
Environment temporary{"TMPDIR", directory.path()};
202+
#else
71203
Environment temporary{"TMPDIR", "/dev/shm"};
204+
#endif
72205
};
73206

74207
inline bool memory_filesystem(std::uint64_t type) {
@@ -101,8 +234,119 @@ inline std::size_t descriptor_count() {
101234
return count;
102235
}
103236

104-
// Inotify observes even immediately unlinked mkstemp files. Looking only at
105-
// directory contents after the run would miss a jailed fallback.
237+
#if defined(PINEFORGE_TEST_SCRATCH_ATTRIBUTION)
238+
// Strict inotify only inside our private TMPDIR; the shared roots are judged
239+
// by successful mkstemp calls whose returned descriptors belong to this process.
240+
class CreationWatch {
241+
public:
242+
CreationWatch() : descriptor_(::inotify_init1(IN_NONBLOCK | IN_CLOEXEC)),
243+
directory_(std::getenv("TMPDIR") ? std::getenv("TMPDIR") : "") {
244+
require(descriptor_.value >= 0, "cannot create scratch creation watch");
245+
constexpr std::string_view prefix = "/dev/shm/pineforge-tmpwatch-";
246+
require(scratch_inspection_path(directory_)
247+
&& directory_.compare(0, prefix.size(), prefix.data()) == 0,
248+
"scratch watch requires the row's private TMPDIR");
249+
std::printf("S4E attributable inspection paths: %s, /tmp, /var/tmp, "
250+
"/proc/self/fd/<returned fd> (readlink names only)\n", directory_.c_str());
251+
struct stat status{};
252+
require(::lstat(directory_.c_str(), &status) == 0 && S_ISDIR(status.st_mode)
253+
&& status.st_uid == ::geteuid() && (status.st_mode & 0777) == 0700,
254+
"scratch TMPDIR is not a private owned directory");
255+
struct statfs memory{};
256+
require(::statfs(directory_.c_str(), &memory) == 0
257+
&& static_cast<std::uint64_t>(memory.f_type) == UINT64_C(0x01021994),
258+
"private TMPDIR must be tmpfs; this row must not skip");
259+
require(::inotify_add_watch(descriptor_.value, directory_.c_str(),
260+
IN_CREATE | IN_MOVED_TO) >= 0,
261+
"cannot watch the private scratch candidate");
262+
auto& registry = mkstemp_registry();
263+
std::lock_guard<std::mutex> lock(registry.mutex);
264+
require(registry.active == nullptr, "scratch attribution watches must not overlap");
265+
registry.active = &trace_;
266+
}
267+
~CreationWatch() {
268+
auto& registry = mkstemp_registry();
269+
std::lock_guard<std::mutex> lock(registry.mutex);
270+
registry.active = nullptr;
271+
}
272+
273+
void collect() {
274+
alignas(inotify_event) std::array<char, 16384> buffer{};
275+
for (;;) {
276+
const auto bytes = ::read(descriptor_.value, buffer.data(), buffer.size());
277+
if (bytes < 0 && errno == EINTR) continue;
278+
if (bytes < 0 && errno == EAGAIN) break;
279+
require(bytes > 0, "cannot read private scratch creation events");
280+
std::size_t offset = 0;
281+
while (offset < static_cast<std::size_t>(bytes)) {
282+
require(static_cast<std::size_t>(bytes) - offset >= sizeof(inotify_event),
283+
"short private scratch creation event");
284+
const auto* event = reinterpret_cast<const inotify_event*>(buffer.data() + offset);
285+
require(!(event->mask & (IN_Q_OVERFLOW | IN_IGNORED)),
286+
"private scratch creation watch was lost");
287+
require(event->len <= static_cast<std::size_t>(bytes) - offset - sizeof(*event),
288+
"short private scratch creation event name");
289+
if (event->mask & (IN_CREATE | IN_MOVED_TO)) {
290+
require(event->len != 0, "private scratch creation lacks a name");
291+
const auto path = directory_ + "/" + event->name;
292+
require(scratch_inspection_path(path), "refused scratch event path by name");
293+
private_paths_.push_back(path);
294+
}
295+
offset += sizeof(*event) + event->len;
296+
}
297+
}
298+
auto& registry = mkstemp_registry();
299+
std::lock_guard<std::mutex> lock(registry.mutex);
300+
require(!trace_.failed, "mkstemp descriptor attribution failed");
301+
}
302+
303+
std::size_t created_count_in(const char* directory) const {
304+
auto& registry = mkstemp_registry();
305+
std::lock_guard<std::mutex> lock(registry.mutex);
306+
require(!trace_.failed, "mkstemp descriptor attribution failed");
307+
std::size_t count = 0;
308+
for (const auto& path : trace_.paths)
309+
if (scratch_name_in(path, directory)) ++count;
310+
for (const auto& path : private_paths_)
311+
if (scratch_name_in(path, directory)) ++count;
312+
return count;
313+
}
314+
bool created_in(const char* directory) const { return created_count_in(directory) != 0; }
315+
bool tmpdir_empty() const { return private_paths_.empty(); }
316+
bool empty() const {
317+
auto& registry = mkstemp_registry();
318+
std::lock_guard<std::mutex> lock(registry.mutex);
319+
require(!trace_.failed, "mkstemp descriptor attribution failed");
320+
return private_paths_.empty() && trace_.paths.empty();
321+
}
322+
void require_removed() const {
323+
auto& registry = mkstemp_registry();
324+
std::lock_guard<std::mutex> lock(registry.mutex);
325+
require(!trace_.failed, "mkstemp descriptor attribution failed");
326+
for (const auto& path : trace_.paths) require_removed_path(path);
327+
for (const auto& path : private_paths_) require_removed_path(path);
328+
}
329+
330+
private:
331+
static bool scratch_name_in(const std::string& path, const char* directory) {
332+
const auto filename = path.find_last_of('/') + 1;
333+
return under_directory(path, directory) && path.compare(filename, 10, "pineforge-") == 0;
334+
}
335+
static void require_removed_path(const std::string& path) {
336+
require(scratch_inspection_path(path), "refused scratch cleanup inspection path by name");
337+
std::printf("S4E owned scratch lstat: %s\n", path.c_str());
338+
struct stat status{};
339+
errno = 0;
340+
require(::lstat(path.c_str(), &status) != 0 && errno == ENOENT,
341+
"run left a named scratch file: " + path);
342+
}
343+
Fd descriptor_;
344+
std::string directory_;
345+
MkstempTrace trace_;
346+
std::vector<std::string> private_paths_;
347+
};
348+
#else
349+
// Other fixture consumers keep their original name-based witness.
106350
class CreationWatch {
107351
public:
108352
CreationWatch() : descriptor_(::inotify_init1(IN_NONBLOCK | IN_CLOEXEC)) {
@@ -166,6 +410,7 @@ class CreationWatch {
166410
std::vector<Created> created_;
167411
bool any_creation_ = false;
168412
};
413+
#endif // PINEFORGE_TEST_SCRATCH_ATTRIBUTION
169414

170415
class Host final : public pineforge::NativeStrategyHost {
171416
public:
@@ -262,3 +507,30 @@ inline int test_main(const char* name, void (*body)()) {
262507
}
263508

264509
} // namespace s4e_tmp
510+
511+
#if defined(PINEFORGE_TEST_SCRATCH_ATTRIBUTION)
512+
// Each opted-in executable has one fixture TU. A strong, visible definition
513+
// resolves both header-inline calls and references from the pineforge archive.
514+
extern "C" __attribute__((visibility("default"))) int mkstemp(char* pattern) {
515+
const int caller_errno = errno;
516+
if (!s4e_tmp::scratch_inspection_path(pattern)) {
517+
s4e_tmp::fail_mkstemp_trace();
518+
errno = EACCES;
519+
return -1;
520+
}
521+
using Function = int (*)(char*);
522+
static const auto real = reinterpret_cast<Function>(::dlsym(RTLD_NEXT, "mkstemp"));
523+
if (!real) {
524+
s4e_tmp::fail_mkstemp_trace();
525+
errno = ENOSYS;
526+
return -1;
527+
}
528+
errno = caller_errno;
529+
const int descriptor = real(pattern);
530+
const int saved_errno = errno;
531+
if (descriptor >= 0) s4e_tmp::record_mkstemp(descriptor, pattern);
532+
errno = saved_errno;
533+
return descriptor;
534+
}
535+
#endif // PINEFORGE_TEST_SCRATCH_ATTRIBUTION
536+
#endif // __linux__

‎tests/test_default_scratch_tmpfs_tmpdir.cpp‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,15 @@
11
// RED FIRST: at 61ab57dac1 the real closed-trade ledger opens its bounded
22
// scratch on TMPDIR's tmpfs and the run fails report_sink_failed/scratch_io.
33
// The native run supplies no sink or scratch-directory argument.
4+
#define PINEFORGE_TEST_SCRATCH_ATTRIBUTION 1
45
#include "default_scratch_tmpfs_fixture.hpp"
56

7+
#if defined(__linux__)
68
namespace {
79
void row() {
810
using namespace s4e_tmp;
9-
PlainEnvironment environment;
1011
require_filesystems();
12+
PlainEnvironment environment;
1113
CreationWatch watch;
1214
const auto before = descriptor_count();
1315
{
@@ -22,12 +24,17 @@ void row() {
2224
"plain native run lost its completed trade or owning report");
2325
require_success(host);
2426
watch.collect();
27+
require(watch.created_in("/var/tmp"),
28+
"native ledger never used its own disk-backed /var/tmp scratch");
2529
watch.require_removed();
2630
}
31+
const auto native_creations = watch.created_count_in("/var/tmp");
2732
exercise_command_scratch(pineforge::detail::ScratchBackingKind::disk);
2833
watch.collect();
2934
require(watch.created_in("/var/tmp"), "default run never used disk-backed /var/tmp");
30-
require(!watch.created_in("/dev/shm") && !watch.created_in("/tmp"),
35+
require(watch.created_count_in("/var/tmp") > native_creations,
36+
"production command scratch lacks its own /var/tmp creation witness");
37+
require(watch.tmpdir_empty() && !watch.created_in("/dev/shm") && !watch.created_in("/tmp"),
3138
"TMPDIR on tmpfs must resolve directly to /var/tmp");
3239
watch.require_removed();
3340
require(descriptor_count() == before, "default run leaked an unlinked scratch descriptor");
@@ -37,3 +44,4 @@ void row() {
3744
int main() {
3845
return s4e_tmp::test_main("test_default_scratch_tmpfs_tmpdir", row);
3946
}
47+
#endif // __linux__

0 commit comments

Comments
 (0)