|
| 1 | +# Issue 611: scope repair validation and operator procedure |
| 2 | + |
| 3 | +Base: `24c68c4a` (v4.23.0). Measured 2026-09-17. |
| 4 | +Decision: ADR-1083, proposed for owner review. This change is not released. |
| 5 | + |
| 6 | +## Regression evidence |
| 7 | + |
| 8 | +The original `remember` regression failed because an agent decision returned |
| 9 | +`is_global=True`. After separating the flags, the affected PostgreSQL suite |
| 10 | +passes 182 tests. SQLite passes 175 tests, with seven PostgreSQL-only tests |
| 11 | +skipped. The sandbox-only SQLite run reports PostgreSQL unavailable; the |
| 12 | +PostgreSQL execution uses a separately created scratch database. The |
| 13 | +reclassification script suite additionally passes 24 tests on PostgreSQL, |
| 14 | +including transaction failure and archive validation paths. |
| 15 | + |
| 16 | +Independent review reproduced an ingestion privilege regression: an `auto` |
| 17 | +write was persisted as `deliberate`, and initialization then granted team |
| 18 | +visibility. Four of eight origin/class reopen cases failed before persisting |
| 19 | +the class. All eight pass after the fix on both backends. SQLite also tests |
| 20 | +an upgrade from the old schema. The PostgreSQL upgrade was exercised on a |
| 21 | +full restore of the pre-change production archive, including the existing |
| 22 | +`current_memories` view. |
| 23 | + |
| 24 | +## Production snapshot experiment |
| 25 | + |
| 26 | +The custom-format production archive was restored in full into an isolated |
| 27 | +PostgreSQL database; `pg_restore` exited zero. Production was not reclassified. |
| 28 | +The script also checks for memories table data and decodes the full archive |
| 29 | +before allowing PostgreSQL apply. This validates readability, not that an |
| 30 | +arbitrary supplied archive belongs to the selected database; the operator |
| 31 | +must create and retain the target's own backup as below. |
| 32 | + |
| 33 | +| Measurement | Result | |
| 34 | +| --- | --- | |
| 35 | +| Current non-benchmark global candidates | 160 | |
| 36 | +| Dry-run changes | 98 | |
| 37 | +| Applied changes on the restored copy | 98 | |
| 38 | +| Changes proposed on the second run | 0 | |
| 39 | +| Retained globals | 62 | |
| 40 | +| Retained globals with unresolved project | 47 | |
| 41 | +| Total rows before and after repair | 47,490 | |
| 42 | + |
| 43 | +All fields other than `is_global`, `is_team_decision`, and `directory_context` |
| 44 | +had identical ordered aggregate hashes before and after repair: |
| 45 | +`f66cfb02e0c7996857ccfe74b311e667`. This includes content, IDs, and supersession. |
| 46 | +The comparison ran before the acceptance hooks, which can record receipts. |
| 47 | + |
| 48 | +## Approved mapping and remaining owner decisions |
| 49 | + |
| 50 | +All 47 originally empty `directory_context` rows also had an empty `domain`. |
| 51 | +On 2026-09-17 the owner explicitly approved assigning memory 4353879 to |
| 52 | +`/Users/cdeust/Developments/japonais-2027`. The operator input is a mappings |
| 53 | +file of the form `{"domains": {}, "memories": {"<id>": "<project root>"}, |
| 54 | +"keep_global_ids": []}`. It names rows of one private store, so it is kept |
| 55 | +outside the repository. |
| 56 | + |
| 57 | +Applied to the already reclassified snapshot, this changes one additional |
| 58 | +row to `is_global=false`, `is_team_decision=true`, with the approved directory. |
| 59 | +The next pass changes zero rows. Across both passes, 99 of the original 160 |
| 60 | +candidates lose global scope; 61 remain global, including 46 unresolved rows. |
| 61 | +The actual auto-recall launcher now excludes the Score audit decision from |
| 62 | +anthropic-partnership and includes it in japonais-2027. Both processes exit |
| 63 | +zero. This verifies the specific 4353879 acceptance signal on the snapshot. |
| 64 | + |
| 65 | +Production remains unreclassified pending deployment of the reviewed code. |
| 66 | +The parent Team Decisions query still returns unresolved row 4356519, whose |
| 67 | +project requires separate owner review. The full SessionStart launcher |
| 68 | +validation timed out after 60 seconds; direct query and subprocess fixture |
| 69 | +tests do not replace final installed-plugin acceptance. |
| 70 | + |
| 71 | +Remaining unresolved IDs, left global without changing their project: |
| 72 | + |
| 73 | +4254394, 4342464, 4343116, 4347543, 4349020, 4349043, 4349560, 4349960, 4349976, 4350007, 4351552, 4351562, 4351771, 4352557, 4352618, 4353539, 4353601, 4353612, 4354421, 4354708, 4355514, 4355539, 4355775, 4356057, 4356262, 4356365, 4356519, 4356548, 4356730, 4357099, 4359407, 4359553, 4359753, 4359754, 4359755, 4359773, 4360000, 4361139, 4362817, 4365424, 4365650, 4367130, 4367241, 4367268, 4367557, 4367684 |
| 74 | + |
| 75 | +The legacy row does not store whether `is_global` was explicit or inferred. |
| 76 | +Use `keep_global_ids` for rows independently known to have been explicitly |
| 77 | +global. The script otherwise applies the existing content detector exactly |
| 78 | +as requested by the reclassification plan. |
| 79 | + |
| 80 | +## Operator procedure |
| 81 | + |
| 82 | +1. Review ADR-1083 and the unresolved ID list. Start with the approved mapping |
| 83 | + file described above. Additional mappings require owner approval. Use a private file |
| 84 | + with verified `domains` and approved `memories` (decimal ID keys) maps |
| 85 | + to existing canonical absolute project directories, plus `keep_global_ids`. |
| 86 | + Empty mapping keys and inferred ownership from prose are rejected. |
| 87 | +2. Deploy the reviewed code and schema before production reclassification. |
| 88 | + Old installations still run the team-to-global backfill at initialization |
| 89 | + and can reverse a data-only repair. Do not mix the repaired data with old |
| 90 | + writers. The owner controls merge, release and installed-plugin updates. |
| 91 | +3. Create a fresh `pg_dump -Fc -f cortex-before-611.dump cortex`. Restore it |
| 92 | + to a separate scratch database and verify successful completion. |
| 93 | +4. Run the script with the explicit target and inspect its JSON report: |
| 94 | + |
| 95 | + ```sh |
| 96 | + python scripts/reclassify_team_scope.py --database-url postgresql:///cortex \ |
| 97 | + --mappings /private/path/scope-mappings.json |
| 98 | + ``` |
| 99 | + |
| 100 | +5. Apply that reviewed mapping with `--apply --backup /private/path/cortex-before-611.dump`. |
| 101 | + Repeat the dry-run and require `change_count: 0`. Preserve the reports and |
| 102 | + backup outside Git. SQLite uses `--sqlite-path` instead of `--database-url`. |
| 103 | +6. Run both acceptance hooks from the updated installed plugin. Require |
| 104 | + 4353879 absent in anthropic-partnership and present in japonais-2027, and |
| 105 | + no japonais-2027 row in the parent's Team Decisions block. Do not claim |
| 106 | + the issue fixed in a release before this. |
| 107 | + |
| 108 | +## Rerun after review (2026-09-17) |
| 109 | + |
| 110 | +The review of #613 found that `classify()` cleared `is_global` on any resolved |
| 111 | +row the detector did not confirm, including rows the ADR-0200 promotion could |
| 112 | +not have produced (no agent context, or no decision content). Those rows were |
| 113 | +made global by an explicit act the script cannot see. The rule is now narrower: |
| 114 | +only a decision written under an agent context loses global scope. Every other |
| 115 | +global row is kept and listed under `unexplained_global_ids`, and the owner can |
| 116 | +clear one through `clear_global_ids` in the mappings file. |
| 117 | + |
| 118 | +The figures above predate that change. The same archive was restored into a |
| 119 | +fresh database and the whole sequence repeated with the approved mapping of |
| 120 | +4353879 supplied from the first pass: |
| 121 | + |
| 122 | +| Measurement | Result | |
| 123 | +| --- | --- | |
| 124 | +| Global candidates | 160 | |
| 125 | +| Dry-run changes | 93 | |
| 126 | +| Applied changes | 93 | |
| 127 | +| Changes proposed on the second run | 0 | |
| 128 | +| Retained globals | 67 | |
| 129 | +| of which unresolved project | 46 | |
| 130 | +| of which kept as not produced by the defect | 6 | |
| 131 | +| Non-benchmark rows before and after | 47,041 | |
| 132 | + |
| 133 | +The six kept rows are 4349818, 4353551, 4353795, 4353809, 4353915 and 4361406. |
| 134 | +Each has an agent context and content the decision cue does not match, so the |
| 135 | +script leaves the call to the owner. The apply step refused to run until the |
| 136 | +`is_team_decision` column existed on the restored copy, as designed. |
| 137 | + |
| 138 | +## Python compatibility |
| 139 | + |
| 140 | +CI Python 3.10 exposed a collection error from importing `typing.Self`, which |
| 141 | +is unavailable in that supported version. The database context manager now |
| 142 | +uses its concrete class as a postponed return annotation, with no new dependency. |
| 143 | + |
| 144 | +## Completion ledger |
| 145 | + |
| 146 | +| Changed behavior or failure path | Evidence | |
| 147 | +| --- | --- | |
| 148 | +| Separate global resolution and team classification | core global-scope and handler team-scope regressions | |
| 149 | +| Trusted ingest class/origin survives reopening | eight-case matrix, both backends | |
| 150 | +| Persist marker and upgrade existing schema/view | SQLite schema-upgrade test; restored PostgreSQL archive | |
| 151 | +| Initialization does not promote team rows to global | backfill tests, including zero second-run row count | |
| 152 | +| Session hooks enforce project predicate before limit | team-project hook tests and real subprocess tests | |
| 153 | +| Agent briefing scopes both selection passes | PostgreSQL team-project regression | |
| 154 | +| Explicit global and detector behavior preserved | global detector tests and cross-project hook fixtures | |
| 155 | +| Domain mapping, approved ID mapping, unresolved rows | script classifier and mapping validation tests | |
| 156 | +| Dry-run, second-run idempotence, history/benchmark exclusion | SQLite script tests and restored snapshot experiment | |
| 157 | +| Transaction rollback and marker refusal | real PostgreSQL and SQLite script tests | |
| 158 | +| Missing/invalid backup, missing table data, full decode | script backup validation tests and full archive restore | |
| 159 | + |
| 160 | +No merge, release or production reclassification is performed by this change. |
0 commit comments