Repository navigation
Commit ae57979
feat(mxc): add Windows ETW-to-OCSF audit trail (NVIDIA#3015)
* feat(mxc): ETW->OCSF audit consumer + Windows OCSF JSONL parity (cp6 P1)
Add a Windows MXC ETW->OCSF audit trail in openshell-driver-mxc: a real-time
Sandboxing-provider ETW consumer that decodes events (TDH), attributes each to
an OpenShell sandbox_id, and maps them to OCSF (lifecycle 6002, config 5019,
process 1007, finding 2004).
cp6 Phase 1 - durable OCSF JSONL audit-file parity with Linux:
- openshell-ocsf: add emit_ocsf_event_routed (populates the event-bridge
thread-local AND stamps sandbox_id+message in one dispatch) plus public
set/clear_current_event; OS-aware device (Device::windows/for_current_os) so
device.os.name reflects the host instead of a hardcoded Linux stub.
- etw_consumer: emit via the routed emit (previously fired a bare info! that
never populated the bridge, so the structured event was dropped).
- openshell-server: install OcsfJsonlLayer over a synchronous daily-rotated
appender (durable under force-kill), gated by OPENSHELL_OCSF_JSON, path via
%PROGRAMDATA%\OpenShell\logs (override OPENSHELL_OCSF_LOG_DIR).
- device.hostname now resolves to the real gateway machine name.
Box-proven on 7F203-MXC-001: JSONL lines == shorthand OCSF rows, all valid
OCSF JSON, per-sandbox attribution intact, disabled state writes nothing.
Signed-off-by: Akber Raza <[email protected]>
* feat(mxc): map remaining Sandboxing ETW events to OCSF
Close the last three ETW->OCSF gaps so the audit trail covers the full
set of events the Sandboxing provider emits (12/12):
- ProcessLaunched -> Process Activity [1007] "Launch" (confirmed start;
carries the real processId/threadId, the twin of CreateProcessInSandbox
which only has the request + command line).
- SandboxProxyConfigured -> Device Config State Change [5019] (the one
network-plane setup event; surfaces proxyPort, "no proxy" when 0).
- SandboxConsoleReferencePlumbed -> Device Config State Change [5019]
(console-handle plumbing).
map_config_state now handles the full config/hardening/setup family and
carries proxyPort/hasConsoleReference/creationFlags as unmapped fields.
Verified on 7F203-MXC-001: 11/12 event types emit OCSF without a proxy
(SandboxProxyConfigured requires proxy config to fire).
Signed-off-by: Akber Raza <[email protected]>
* fix(mxc): seed ETW attribution under registry lock + Device tests
Address CodeRabbit review on !31:
- Prevent stale ETW attribution on a delete/launch race: register the
wxc-exec pid while holding the registry lock, and bail if the sandbox
entry is already gone. Previously the attribution key could be seeded
after `delete` had removed the sandbox, leaving a stale key that could
misroute later Sandboxing ETW events to a dead sandbox_id. Lock order
(registry -> attribution) matches the delete path, so no deadlock.
- Add unit tests for the new Device::windows and Device::for_current_os
constructors to harden Windows/Linux OCSF device parity.
Signed-off-by: Akber Raza <[email protected]>
* fix(mxc-etw): buffer+replay racing events and harden attribution keys
Addresses two ETW->OCSF attribution review items (Shailendra #1, #2).
#2 early-event loss: ETW delivers the sandbox create/config burst the instant wxc-exec starts, which can beat the driver's register_launch (now under the registry lock post-Ready). process_event previously dropped anything unresolved, losing the racing burst. Add a bounded, time-bounded pending buffer (PENDING_MAX=4096, PENDING_TTL=5s): unresolved events are held and replayed once attribution lands, aged-out ones dropped. Consumer switched to a timed recv_timeout(200ms) so the buffer is re-driven after each event and on a tick. Emit path factored into shared emit_resolved().
#1 attribution collisions: a Windows PID is recycled after exit and a command line is commonly identical across sandboxes. register_launch now rebinds by_pid on reuse and clears the stale last_pid_sid hint (warns if the PID still pointed at a different, leaked sandbox); command line is held in by_cmd only while unique and demoted to a new ambiguous_cmds set on a second owner, so a duplicate command refuses to resolve rather than misroute.
Unit tests: buffer replay (direct + cross-link), buffer bound, PID-reuse rebind, duplicate-cmd non-resolution. Box-verified on 7F203-MXC-001 (5 sandboxes, identical cmd -> 5 isolated sandbox_ids, 50/50 OCSF/JSONL, BuffersLost=0).
Signed-off-by: Akber Raza <[email protected]>
* docs(mxc-etw): note cmd_line is captured raw with no privacy filtering
Review item NVIDIA#3 (Shailendra): add a PRIVACY NOTE on map_process_launch stating cmd_line is copied verbatim into OCSF process.cmd_line with no redaction, so secrets/PII on a command line land unredacted in the durable audit trail (deliberate audit-fidelity trade-off; treat the log as sensitive). Redaction is owned by an upstream privacy layer, not this path; no general audit-output PII scrubber exists today (openshell_core::secrets [CREDENTIAL] redaction is scoped to the proxy HTTP-target logging, a separate egress path).
Signed-off-by: Akber Raza <[email protected]>
* fix(mxc-etw): open ETW trace on caller thread so start_session reports real status
Review item NVIDIA#4 (Shailendra): start_session previously returned Ok(EtwSession) as soon as the pump thread was spawned, but OpenTraceW ran later inside that thread; if it failed we still handed back a live-looking session and logged 'consumer started' (silent failure = false audit coverage).
Split the two Win32 calls instead of adding a channel handshake (avoids any lost-wakeup/hang risk): the quick, synchronous OpenTraceW now runs on the caller thread (open_trace), and only the blocking ProcessTrace runs on the pump thread (run_trace). start_session returns Err if OpenTraceW fails (reclaiming the boxed Sender so the consumer disconnects, stopping the session, joining the consumer) and returns Ok/logs 'started' only once capture is genuinely open. Opened handle + LoggerName buffer + boxed Sender are carried to the pump via a Send OpenedTrace so they outlive ProcessTrace.
Box-verified on 7F203-MXC-001: consumer started=True, failed-to-start=False, 50 OCSF rows / 50 JSONL, BuffersLost=0 (no regression to capture/emit).
Signed-off-by: Akber Raza <[email protected]>
* fix(mxc-etw): guard pending-event replay against PID recycling
CodeRabbit flagged that drain_resolved() re-resolved buffered events
against the live by_pid map, so if Windows recycled a wxc-exec PID within
PENDING_TTL a stale event from the dead sandbox could be emitted under the
new owner.
Stamp each by_pid registration with its Instant and add resolve_replay(),
used only on the buffered/replay path. It (a) never falls back to the
recycle-/ambiguity-prone by_cmd or last_pid_sid keys, and (b) trusts a PID
match only when the registration is not newer than the buffered event by
more than REPLAY_PID_GRACE (2s) - a recycled PID's registration lands well
outside that window, so the stale event ages out instead of misattributing.
The legitimate #2 seed race (registration lands ~immediately) still replays.
Adds unit tests for the recycle-refusal, in-grace acceptance, and
weak-fallback exclusion.
Signed-off-by: Akber Raza <[email protected]>
* fix(mxc-etw): surface unexpected ProcessTrace termination (review NVIDIA#4)
start_session already returns Err on OpenTraceW failure (runs on the
caller thread since e41a770), closing the first half of Shailendra's NVIDIA#4.
This closes the second half: ProcessTrace's result was discarded, so if
capture died mid-run the backend had no way to know.
Add a shared CaptureHealth (stopped/stopping/exit_code) between the pump
thread and EtwSession. run_trace now records ProcessTrace's WIN32_ERROR
and, when the pump returns without a deliberate stop, logs at ERROR that
MXC OCSF capture is no longer running. EtwSession::stop() sets `stopping`
before teardown so a normal shutdown isn't misreported, and
EtwSession::is_capture_alive() exposes the state for status/diagnostics.
Box-verified on 7F203-MXC-001: 5 sandboxes, 50 attributed OCSF rows,
JSONL parity 50/50, BuffersLost=0, clean start/stop (no false failure).
Signed-off-by: Akber Raza <[email protected]>
* feat(mxc-ocsf): add ETW->OCSF audit-trail example kit; fix proxy-configured message
Add a runnable OCSF audit-trail example under examples/ (run-ocsf-audit.ps1,
mxc-ocsf-audit.toml, ocsf-audit.yaml, README) that spins up sandboxes with the
in-process ETW consumer and egress proxy on, emitting a full OCSF JSONL audit
trail across all four classes (6002/5019/1007/2004).
Fix SandboxProxyConfigured mapping to log "MXC sandbox proxy configured" instead
of a misleading "(no proxy)" when the provider reports proxyPort=0; the event's
presence already indicates proxy configuration. Verified on-box: 26 events, all
mapped ETW event types present.
Signed-off-by: Akber Raza <[email protected]>
* feat(mxc-ocsf): clearer audit report + client-safe run-ocsf-audit.ps1
Improve the ETW to OCSF audit-trail example output and make it safe to ship.
Report:
- Add an event-type coverage count ("N of M expected event types fired");
the denominator auto-adjusts (8 with proxy on, 7 with -NoProxy).
- Split the checklist into expected event types vs anomaly findings
(ActivityError/FallbackError), which are reported separately and not
counted toward coverage (a clean run may emit none).
- Verdict is now coverage-based (all expected types must fire) instead of
the looser "at least 3 OCSF classes".
- Call out the absolute path to the durable OCSF JSONL log prominently.
Client-safety:
- Default -ShareOut to empty (no auto-copy); pass -ShareOut a UNC path to
opt in. Removes a hardcoded internal share path from a published example.
- Drop internal-team wording ("Hand that zip back for evaluation", "BUNDLE:")
in favor of neutral "Results bundle:".
- Update README-ocsf-audit.txt to match the opt-in -ShareOut behavior.
Verified on both MXC boxes: 7F203-MXC-001 (base-container) -> PASS, 8 of 8
event types, 26 OCSF events across 4 classes; 7F203-MXC-003 (AppContainer
fallback) -> reduced set as expected, clean output.
Signed-off-by: Akber Raza <[email protected]>
* fix(mxc): configure OCSF audit workloads per sandbox
- remove unsupported gateway-scoped workload fields from the shipped MXC audit example.
- build the command, working directory, and filesystem grant from each run's ShareDir
- pass the workload through --driver-config-json.
- preserve the host CONNECT proxy configuration and conditional audit coverage for the future host_connect_proxy merge
- require the workload output when determining the audit verdict.
Signed-off-by: Akber Raza <[email protected]>
* fix(mxc): omit command arguments from OCSF audit logs
- record only the executable basename for MXC CreateProcessInSandbox audit events
- leave process.cmd_line unset so workload arguments cannot reach shorthand or JSONL logs
- cover tokens, passwords, signed URLs, and PII with a secret-leak regression test
- update the audit example, architecture guidance, and published logging documentation
- preserve ETW attribution and future host_connect_proxy enforcement behavior
Signed-off-by: Akber Raza <[email protected]>
* feat(etw): enhance ETW session management with distinct naming for concurrent gateways
* fix(etw): bound the audit queue during overload
- replace the unbounded ETW callback channel with count- and byte-bounded buffering
- keep the ETW callback non-blocking and count records rejected during overload
- emit immediate, rate-limited warnings that identify resulting audit coverage gaps
- make the audit example fail when queue overload causes dropped ETW records
- cover stalled consumers, oversized events, and warning throttling with unit tests
Signed-off-by: Akber Raza <[email protected]>
* fix(etw): harden sandbox audit attribution
- remove command-line and persistent per-PID fallback keys from live and replay resolution
- retire the driver-owned wxc-exec PID before publishing child completion
- retain established identity, activity, and correlation-vector links only for the five-second late-event window
- prevent buffered records from crossing rapid PID retirement and reuse boundaries
- add resolver and lifecycle coverage and document the attribution trust boundary
Signed-off-by: Akber Raza <[email protected]>
* chore(mxc): address rebase follow-ups
Signed-off-by: Drew Newberry <[email protected]>
* fix(mxc): align OCSF audit example with driver config
- remove unsupported egress proxy settings
- stop requiring the unavailable proxy audit event
- update example documentation for supported event coverage
Signed-off-by: Akber Raza <[email protected]>
* fix(etw): redact command-line secrets in DecodedEtwEvent summary
* fix(etw): enhance PID resolution and event attribution logic for ETW records
* fix(ocsf): restrict gateway-local JSONL sink to Windows/MXC path with opt-in configuration
* address rebase issues
* fix(mxc): address ETW audit review feedback
Signed-off-by: Drew Newberry <[email protected]>
* fix(mxc): fail closed across ambiguous PID reuse
Signed-off-by: Drew Newberry <[email protected]>
* fix(mxc): bind ETW attribution to process generation
Signed-off-by: Drew Newberry <[email protected]>
---------
Signed-off-by: Akber Raza <[email protected]>
Signed-off-by: Drew Newberry <[email protected]>
Co-authored-by: Jamie King <[email protected]>
Co-authored-by: Drew Newberry <[email protected]>1 parent 0569c3a commit ae57979
21 files changed
Lines changed: 3568 additions & 31 deletions
File tree
- architecture
- crates
- openshell-driver-mxc
- examples
- src
- openshell-ocsf/src
- builders
- objects
- tracing_layers
- openshell-server
- src
- docs
- observability
- reference
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
57 | 60 | | |
58 | 61 | | |
59 | 62 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
389 | 389 | | |
390 | 390 | | |
391 | 391 | | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
18 | 22 | | |
19 | 23 | | |
20 | 24 | | |
| |||
26 | 30 | | |
27 | 31 | | |
28 | 32 | | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
29 | 38 | | |
30 | 39 | | |
31 | 40 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| 45 | + | |
45 | 46 | | |
46 | 47 | | |
47 | 48 | | |
| |||
58 | 59 | | |
59 | 60 | | |
60 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
61 | 89 | | |
62 | 90 | | |
63 | 91 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
0 commit comments