|
| 1 | +# DeepSeek Harness: runtime adoption audit for RedCode |
| 2 | + |
| 3 | +Date: 2026-08-14 |
| 4 | +Query: Revisit the DeepSeek Harness evaluation, verify which principles are active in RedCode's shipped runtime, and close the safest high-value gaps. |
| 5 | +Scope: Official Harness repository at commit `47f943859bef60e4160492346772ded9b24f765a`; RedCode's real Location and internal-plugin boot path; composition, inspection, lifecycle, invariants, session durability, tool policy, and testing. This supplements rather than replaces `2026-08-14-deepseek-harness.md`. |
| 6 | + |
| 7 | +## Executive Summary |
| 8 | + |
| 9 | +The first RedCode adoption wave captured the right architecture but left one consequential implementation gap: `CordisPluginHost` was covered by isolated tests while the production internal-plugin boot still called `PluginV2.add` directly in a forked Effect. That meant the shipped runtime did not receive the profile ownership, ordered activation, awaited readiness, transactional rollback, or active-profile diagnostics described by the ADR. |
| 10 | + |
| 11 | +This audit closes that gap. Internal plugins are now one named `internal` profile applied through the Cordis host. Boot awaits activation, then runs the Location-scoped invariant registry. A typed `LayerNode.inspect` projection exposes the effective Location service graph, including replacements and dependency order, without introducing a second handwritten inventory. A real Location composition test proves the shipping boot path activates exactly the declared built-ins and that its invariants pass. |
| 12 | + |
| 13 | +The result adopts the highest-leverage Harness principles without importing its product topology: reversible ownership, inspectable composition, ordered activation, runtime architectural checks, and real-entry-path testing. Effect remains RedCode's service/resource kernel, and SessionV2 remains the durable execution authority. |
| 14 | + |
| 15 | +## Official Sources |
| 16 | + |
| 17 | +- [DeepSeek Harness repository](https://github.com/deepseek-ai/deepseek-harness) — official repository and developer-preview status. |
| 18 | +- [Architecture](https://github.com/deepseek-ai/deepseek-harness/blob/47f943859bef60e4160492346772ded9b24f765a/docs/architecture.md) — plugin tree, ordered profiles, event domains, session log, and extension points. |
| 19 | +- [Capability seams](https://github.com/deepseek-ai/deepseek-harness/blob/47f943859bef60e4160492346772ded9b24f765a/docs/capability-seams.md) — generated definition/provider/consumer inventory. |
| 20 | +- [Tool execution pipeline](https://github.com/deepseek-ai/deepseek-harness/blob/47f943859bef60e4160492346772ded9b24f765a/docs/tool-execution-pipeline.md) — layered policy, monotonic guards, finalization, and durable tool outcomes. |
| 21 | +- [Runtime invariants](https://github.com/deepseek-ai/deepseek-harness/blob/47f943859bef60e4160492346772ded9b24f765a/docs/subsystems/invariants.md) — package-owned checks, scoped registration, attribution, and lifecycle. |
| 22 | +- [Testing policy](https://github.com/deepseek-ai/deepseek-harness/blob/47f943859bef60e4160492346772ded9b24f765a/docs/testing.md) — real composition, external ground truth, semantic assertions, and lifecycle cleanup tests. |
| 23 | +- [Persistence catalog](https://github.com/deepseek-ai/deepseek-harness/blob/47f943859bef60e4160492346772ded9b24f765a/docs/persistence-catalog.md) — durable versus projection-producing events. |
| 24 | +- [Vendored lifecycle hardening](https://github.com/deepseek-ai/deepseek-harness/blob/47f943859bef60e4160492346772ded9b24f765a/vendor/README.md) — transactional reconciliation, serialized mutation, rollback, and configuration diagnostics. |
| 25 | +- [Post-mortems](https://github.com/deepseek-ai/deepseek-harness/blob/47f943859bef60e4160492346772ded9b24f765a/docs/postmortem/README.md) — process guardrails derived from escaped failures. |
| 26 | + |
| 27 | +## Implementation Coverage Audit |
| 28 | + |
| 29 | +| Harness principle | Before this audit | After this audit | Evidence / boundary | |
| 30 | +|---|---|---|---| |
| 31 | +| Every extension has a reversible owner | PluginV2 effects had Effect scopes; Cordis ownership existed only in isolated host use | Internal production plugins are mounted by Cordis fibers that await Effect teardown | `packages/core/src/plugin/internal.ts`, `packages/core/src/plugin/cordis.ts` | |
| 32 | +| Boot is an ordered, named composition | Built-ins were an imperative sequence of direct `add` calls | Built-ins form the ordered `internal` profile | `PluginInternal.builtInIDs` and live Location test | |
| 33 | +| “Ready” means activation settled | Internal boot was forked in the background | The Location layer awaits profile application before boot succeeds | `PluginInternal.boot` span | |
| 34 | +| Failed composition restores the prior tree | Host rollback existed but did not protect internal boot | Production composition uses that transaction boundary | Cordis host rollback tests plus real boot test | |
| 35 | +| Running architecture is inspectable | Plugin IDs were listable; service topology remained compile-only | `LayerNode.inspect` derives effective service nodes, dependencies, tags, and replacements | `LocationServiceMap.serviceInventory` | |
| 36 | +| Architectural checks belong to the owner | Registry and PluginV2 check existed | Boot now executes registered invariants after the profile settles | `RuntimeInvariant.Service.run` in internal boot | |
| 37 | +| Test the real entry path | Host tests used a test layer | A temporary Location boots all real internal plugins and checks exact inventory | `packages/core/test/location-layer.test.ts` | |
| 38 | +| Durable facts differ from live hooks | Already present in SessionV2 design | Preserved unchanged | Durable admission, exact retries, delivery modes, and Location-scoped runner remain authoritative | |
| 39 | + |
| 40 | +## Findings and Decisions |
| 41 | + |
| 42 | +### 1. Composition abstractions must protect the production path |
| 43 | + |
| 44 | +An unused host is not an adopted architecture. The decisive test is whether the normal Location boot can bypass the ownership and transaction boundary. RedCode could; now it cannot for internal plugins. The profile declaration is also the inventory used by the test, so adding a built-in changes the expected production composition automatically. |
| 45 | + |
| 46 | +### 2. Inspection should be derived from executable structure |
| 47 | + |
| 48 | +Harness generates capability and Cordis catalogs from its source. RedCode's closest source of truth is `LayerNode`, which already encodes service identities, tags, dependencies, and legal replacements. `LayerNode.inspect` walks the effective graph with the same replacement semantics as compilation and returns typed data. This is safer than maintaining a parallel diagram or JSON manifest by hand. |
| 49 | + |
| 50 | +The next increment can render this data as TOON or human-readable text in a diagnostics command. Serialization belongs at that boundary; the Core graph API should remain typed data. |
| 51 | + |
| 52 | +### 3. Invariants are useful only at a meaningful boundary |
| 53 | + |
| 54 | +Registration alone does not protect boot. Running checks after profile activation turns inventory consistency into a readiness condition: an invalid runtime cannot silently advertise itself as ready. RedCode's smaller registry intentionally differs from Harness's configurable package filters; allow/block configuration is not needed until production diagnostics demonstrate a cost or compatibility reason. |
| 55 | + |
| 56 | +### 4. Harness's tool pipeline is a design checklist, not drop-in code |
| 57 | + |
| 58 | +The official pipeline separates pre-policy, monotonic owner guards, around-dispatch behavior, post-processing, definition-owned finalization, immutable observation, and the single durable model-facing result. RedCode should audit its existing tool path against that ordering before adding hooks. It should not graft event stages onto SessionV2 without proving that every model-visible result remains durable and replayable. |
| 59 | + |
| 60 | +### 5. Tests must assert semantic success |
| 61 | + |
| 62 | +Harness's filesystem-tool post-mortem is directly applicable: snapshots can faithfully record a broken composition. RedCode's direct-chat e2e therefore asserts the actual draft route, visible composer, absence of persistent session writes, the `redcode` accessible wordmark, and absence of the legacy OpenCode mark. The new runtime test asserts exact active plugin IDs rather than merely observing that boot did not throw. |
| 63 | + |
| 64 | +## Adoption Matrix |
| 65 | + |
| 66 | +### Adopted now |
| 67 | + |
| 68 | +- Direct-to-chat draft entry, with the old splash/mini-input step removed from the default route. |
| 69 | +- RedCode wordmark and brand palette in the top-level shell. |
| 70 | +- Cordis as the outer reversible owner of real internal PluginV2 composition. |
| 71 | +- Ordered named profile activation with serialized replacement, awaited teardown, and rollback. |
| 72 | +- Stable active plugin inventory. |
| 73 | +- Package-owned, scoped runtime invariants executed as part of boot readiness. |
| 74 | +- Derived inspection of the effective Location service dependency graph. |
| 75 | +- Real-composition and lifecycle tests in addition to isolated host tests. |
| 76 | +- Existing SessionV2 separation of durable prompt admission from model execution. |
| 77 | + |
| 78 | +### Adopt after a dedicated design |
| 79 | + |
| 80 | +- Schema-owned declarative profiles with stable row IDs, provenance, redaction, and one pure patch resolver shared by boot, inspection, reload, and tests. |
| 81 | +- A `dump-config`/architecture command rendered from typed inventories, preferably TOON for machine-consumed structured output. |
| 82 | +- Transactional file reload using candidate validation, a serialized mutation queue, atomic commit, rollback, and watcher teardown tests. |
| 83 | +- A generated definition/provider/consumer capability catalog backed by package metadata and `LayerNode` edges. |
| 84 | +- More owner-specific invariants, especially model-visible-input reconstructability and capability completeness. |
| 85 | +- A formal tool-pipeline audit covering policy ordering, immutable final outcomes, cancellation, and durable result adjacency. |
| 86 | + |
| 87 | +### Do not transplant |
| 88 | + |
| 89 | +- Replacing Effect layers with Cordis services: this would duplicate RedCode's dependency kernel and weaken compile-time service requirements. |
| 90 | +- Replacing SessionV2 with Harness's session implementation: RedCode's durable inbox admission, retry reconciliation, delivery vocabulary, and serialized coordinator are product invariants. |
| 91 | +- Dynamic model-authored plugins before provenance, permissions, signature, persistence, and rollback semantics exist. |
| 92 | +- Background jobs or goals without durable Protocol identity, recovery, cancellation, and placement semantics. |
| 93 | +- Harness YAML or JavaScript-expression configuration verbatim. RedCode needs Schema ownership and must avoid executable configuration crossing trust boundaries. |
| 94 | + |
| 95 | +## Risks and Gotchas |
| 96 | + |
| 97 | +- Awaiting internal profile activation changes readiness timing intentionally. Startup regressions should be measured, but returning before required plugins settle would violate the adopted lifecycle contract. |
| 98 | +- RedCode uses published `@deepseek-ai/cordis`, while Harness vendors and hardens its framework. Only behavior covered by RedCode tests should be relied upon. |
| 99 | +- An inventory is diagnostic evidence, not an authorization surface. Runtime code must continue to depend on typed services and Protocol contracts. |
| 100 | +- Graph inspection may contain repeated names when intentionally distinct branch-local implementations share identity; a future renderer must preserve enough provenance to avoid false deduplication. |
| 101 | +- A successful snapshot is not semantic proof. Tests must also assert required tools, routes, durable records, or externally observed state. |
| 102 | + |
| 103 | +## Recommended Next Steps |
| 104 | + |
| 105 | +1. Land the runtime boot integration and graph inspection in this change. |
| 106 | +2. Add a read-only diagnostics surface that renders the active plugin profile and Location service inventory without secrets. |
| 107 | +3. Specify the profile Schema and pure patch algebra before adding files or HMR. |
| 108 | +4. Audit tool execution ordering and durable results against the official Harness pipeline. |
| 109 | +5. Add SessionV2-owned reconstructability invariants only where authoritative durable and projected data can be compared without new cross-package dependencies. |
| 110 | + |
| 111 | +## Version Notes |
| 112 | + |
| 113 | +- The official repository was rechecked on 2026-08-14; `master` still resolved to `47f943859bef60e4160492346772ded9b24f765a`, the same commit used by the original report. |
| 114 | +- Harness is explicitly a developer preview and expects breaking changes. Principles are adopted behind RedCode-owned contracts rather than treated as upstream API stability promises. |
| 115 | +- RedCode is on release version `0.2.0` before the changes in this supplemental audit; the repository's Version PR remains the only writer of future release numbers. |
0 commit comments