Skip to content

feat(wiki): persist a claim's event time in wiki.claim_events (#717, slice 2) - #894

Merged
github-actions[bot] merged 5 commits into
mainfrom
fix/717-event-time-persistence
Nov 6, 2026
Merged

github-actions[bot] merged 5 commits into
mainfrom
fix/717-event-time-persistence

Conversation

@cdeust

@cdeust cdeust commented Nov 5, 2026

Copy link
Copy Markdown
Owner

Summary

Slice 2 of #717: wiki.claim_events persists ClaimEvent.event_time. Slice 1 (#875) defined the EventTime contract, but the table had no column for it, so a stored claim kept only extracted_at. Root cause: the contract stopped at the model; the insert path (_claim_to_dict → insert_claim_events) and both schemas dropped the field.

  • New column event_time, the JSON of EventTime: JSONB in PostgreSQL, JSON in SQLite (decltype → json.loads). JSON rather than a TIMESTAMPTZ pair: TIMESTAMPTZ normalises to UTC and erases both the naive/aware distinction and the offset the source stated, which slice 1 requires to be kept.
  • One-shot migration for older stores: ALTER TABLE … ADD COLUMN IF NOT EXISTS in MIGRATIONS_DDL (PG), an entry in sqlite_schema.MIGRATIONS (SQLite). Existing rows read back {"kind": "unknown"}, never a date taken from extracted_at.
  • wiki_extract._claim_to_dict writes claim.event_time.model_dump(mode="json").

Refs #717 (remaining slices: extracting stated dates in core/claim_extractor, then carrying the field through synthesis and retrieval).

Type of change

  • New feature (non-breaking change which adds functionality)

Test plan

  • New tests_py/infrastructure/test_claim_event_time_persistence.py, parametrised SQLite / PostgreSQL:
    • unknown, point (naive, day) and both interval forms (aware +02:00 minute bounds; open start, year) round-trip equal, offsets and naivety preserved;
    • a 1969 event extracted in 2026 keeps 1969;
    • a row written with no event time reads back unknown;
    • an older SQLite store without the column gains it, existing rows unknown.
  • Locally (SQLite): 4 passed, 3 PostgreSQL variants skipped (no pgvector in this environment). The same 4 tests fail on main (KeyError: 'event_time').
  • 101 neighbouring test files (schema, wiki, claim, parity, forget, migration, translation): 1345 passed, 35 skipped, 1 failed: tests_py/core/test_wiki_coverage_dashboard.py::test_unreadable_page_is_skipped_not_fatal, which fails the same way on main here (the environment runs as root, so a chmod-000 page is still readable).
  • ruff check / ruff format --check on the changed files: clean. python scripts/check_craftsmanship.py: OK.

Audit notes

  • Not verified here: the PostgreSQL variants and the migration on a live PostgreSQL store (CI), the full suite and pyright (torch cannot be downloaded in this environment).

Coding-standards compliance

  • Layer direction preserved (infrastructure imports nothing from core; the handler stays the composition root).
  • Every new constant carries a source: annotation.

Breaking changes

None: the column has a default, and callers that pass no event_time store unknown.

Reviewer checklist

  • CHANGELOG.md updated under Unreleased / Added.

Generated by Claude Code

…slice 2)

ClaimEvent.event_time (slice 1, #875) existed only in memory: the table had
no column for it, so a stored claim kept extracted_at and lost the time of
the event it describes. The column holds the JSON of EventTime (JSONB in
PostgreSQL, JSON in SQLite) so a naive bound stays naive and an aware bound
keeps its stated offset; a TIMESTAMPTZ pair would normalise both to UTC.
Older stores migrate once, existing rows reading back unknown.

Refs #717

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018kyZYunvyM3xpNmz9DLUT7

cdeust commented Nov 5, 2026

Copy link
Copy Markdown
Owner Author

CI autopilot : before-and-after (Supersession deletion parity) échoue sur une panne d'infrastructure du workflow, pas sur le code de cette PR.

  • Run 42094398825, job 127480549016 : les 15 tests de test_supersession_delete_parity.py passent (« 15 passed »), puis l'étape de mutation de scripts/ci_unit.sh (ajoutée par ci: gate pull requests on mutation testing of the functions they change (ADR-1122) #872, ADR-1122) échoue avec error: Failed to spawn: mutmut — No such file or directory pour wiki_extract.py et pg_store_wiki_claims.py. Le verdict « surviving mutants » qui suit n'est pas un vrai résultat de mutation : aucun mutant n'a tourné.
  • Cause : mutmut vit dans le groupe de dépendances dev (pyproject.toml, [dependency-groups]). .github/actions/uv-locked-install lance uv sync … --no-default-groups $UV_SET, et ce workflow passe --extra dev --extra sqlite --extra codebase --extra postgresql --group lint, sans --group dev. Le job unit de Pipeline (ci.json) lance uv sync --locked sans --no-default-groups, donc lui a mutmut. Ce workflow ne se déclenche que sur les PR qui touchent sqlite_schema.py/sqlite_store.py ; c'est son premier run depuis ci: gate pull requests on mutation testing of the functions they change (ADR-1122) #872.
  • Correctif proposé (même ligne dans memory-exact-review.yml, qui appelle aussi ci_unit.sh). Je ne peux pas le pousser, car le token n'a pas la permission workflows :
--- a/.github/workflows/supersession-delete-parity.yml
+++ b/.github/workflows/supersession-delete-parity.yml
@@
       - name: Install the locked unit and lint environment
         uses: ./.github/actions/uv-locked-install
         with:
           environment: ${{ github.workspace }}/.venv
-          set: --extra dev --extra sqlite --extra codebase --extra postgresql --group lint
+          set: --extra dev --extra sqlite --extra codebase --extra postgresql --group lint --group dev

À décider par un humain : appliquer ce diff sur main (les deux workflows), ou retirer before-and-after des PR qui ne touchent pas la suppression de supersession. Pas de pin d'image, pas de contournement dans scripts/ : faire sauter la mutation quand mutmut manque serait un repli silencieux. Je n'ai pas pu vérifier localement que la mutation tue tous les mutants des deux fonctions modifiées : mutmut a besoin de la pile complète, et torch n'est pas téléchargeable ici.


Generated by Claude Code

@cdeust cdeust added the autopilot:hold CI autopilot stood down: a human decision is required label Nov 5, 2026 — with Claude

cdeust commented Nov 5, 2026

Copy link
Copy Markdown
Owner Author

CI autopilot : unit (run 42094398845) échouait sur le gate de mutation (ADR-1122) à cause d'un vrai trou de test de cette PR.

  • Le gate mute toute la fonction modifiée wiki_extract._claim_to_dict : 18 mutants survivaient, car aucun test ne fixait les clés et valeurs autres que event_time (session_id, claim_type, entity_ids, evidence_refs, confidence, supersedes), ni la forme exacte du model_dump. pg_store_wiki_claims.py sortait à 0 survivant dans le même run.
  • Correctif c467981a (fixup! du commit de la PR) : un test fige le dictionnaire complet produit pour un claim dont tous les champs sont renseignés, y compris event_time sérialisé en JSON et sans borne None.
  • Vérifié localement : GITHUB_BASE_REF=main python scripts/ci_mutation_diff.py donne « none — 0 surviving mutants » pour wiki_extract.py. pg_store_wiki_claims.py n'est pas vérifiable ici, ses tests PostgreSQL étant ignorés faute de pgvector ; la CI fera foi. Le hold reste posé pour before-and-after (commentaire précédent, correctif de workflow à appliquer par un humain).

Generated by Claude Code

github-actions Bot pushed a commit that referenced this pull request Nov 6, 2026
#910)

ci_unit.sh ends with the mutation gate and mutmut lives in the dev dependency
group; uv-locked-install syncs with --no-default-groups, so memory-exact-review
and supersession-delete-parity failed with "Failed to spawn: mutmut" before any
mutant ran (#894). A test now requires --group dev in every workflow that runs
the unit suite through uv-locked-install.

cdeust commented Nov 6, 2026

Copy link
Copy Markdown
Owner Author

CI autopilot : hold levé. main porte #910 (13190fa), qui synchronise le groupe dev dans les workflows qui lancent ci_unit.sh, et la fusion de main du 2026-11-06 l'a apporté sur cette branche.

  • Run 42220863506 (before-and-after, head 4e67bc43) : mutmut tourne désormais. Il reste un mutant survivant, x_insert_claim_events__mutmut_12, qui remplace c.get("session_id", "") par c.get(None, ""). Aucun test de la sélection ne relisait session_id après l'insertion.
  • Correctif cbd5c3c5 (fixup! du commit de la PR, poussé sans réécrire l'historique) : un test insère un claim avec session_id et relit la colonne, sous SQLite et PostgreSQL.
  • Vérifié localement : GITHUB_BASE_REF=main python scripts/ci_mutation_diff.py tue le mutant 12, et wiki_extract.py sort à 0 survivant. Non vérifié ici : les mutants de pg_store_wiki_claims.py que seuls les tests PostgreSQL tuent, faute de pgvector ; la CI fera foi.

Generated by Claude Code

@cdeust cdeust removed the autopilot:hold CI autopilot stood down: a human decision is required label Nov 6, 2026

cdeust commented Nov 6, 2026

Copy link
Copy Markdown
Owner Author

CI autopilot : before-and-after (run 42223836157, head cbd5c3c5) : 13 mutants survivaient dans insert_claim_events, tous de vrais trous de test.

  • Aucun test de la sélection ne relisait claim_type, confidence, supersedes ni evidence_refs après l'insertion. Les valeurs par défaut ("", "assertion", 0.5, []) et la troncature de text à 1900 caractères n'étaient pas vérifiées non plus. Mon test précédent ne couvrait que session_id, d'où le décalage avec le run d'avant.
  • Correctif 4e14e91b (fixup!) : un test insère une ligne minimale puis une ligne complète qui supersède la première, et relit chaque colonne : défauts, texte de 1950 caractères tronqué à 1900, supersedes égal à l'id renvoyé. La troncature et l'id renvoyé tuaient aussi le mutant 78 (out.append(None)).
  • Vérifié localement : GITHUB_BASE_REF=main python scripts/ci_mutation_diff.py donne « 2 file(s), 0 failed, 0 untested » sous SQLite. Les variantes PostgreSQL n'ont pas pu tourner ici ; la CI fera foi.

Generated by Claude Code

@cdeust
cdeust marked this pull request as ready for review November 6, 2026 22:32
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T22:33:27.441828Z 4e14e91 Draft marked ready
🔒 Security Review ✅ Completed 2026-11-06T22:34:55.019567Z 4e14e91 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@claude

claude Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Claude finished @cdeust's task in 2m 54s —— View job


Review of PR #894 (review 1 of 3)

  • Gathered the diff and context
  • Ran five reviews: quality, performance, tests, docs and security
  • Posted the summary and verdict: review comment

None of the five reviews found a Blocker. Three Notes are posted, none needed before merge:

  • The slice 1 line in the CHANGELOG is now out of date.
  • The PostgreSQL test variants leave orphan claim rows behind.
  • The DDL defaults are never exercised on a fresh store.

Verdict: approve

@claude

claude Bot commented Nov 6, 2026

Copy link
Copy Markdown

Review 1 of 3 (ADR-1119): PR #894, head 4e14e91

I ran five reviews: quality, performance, tests, docs and security. None of them found a Blocker. Here is what I traced:

  • PostgreSQL schema order: WIKI_SCHEMA_DDL runs before MIGRATIONS_DDL in get_all_ddl() (pg_schema.py:1967 then :1974). The ADD COLUMN IF NOT EXISTS with a constant default changes only the table definition on PG 11+, so it does not rewrite existing rows.
  • SQLite migration: it runs after the wiki DDL. The duplicate-column error on a fresh store is the existing caught path (sqlite_store.py:172). The JSON declared type gets json.loads through PARSE_DECLTYPES (sqlite_store.py:97).
  • Round trip: model_dump(mode="json", exclude_none=True) validated back gives the same EventTime. A naive bound stays naive and an aware one keeps its offset.
  • Other readers: only get_claims_for_memory uses SELECT *, and its callers read keys by name.
  • Security: inserts use bound parameters, and the SQLite ALTER is built only from the constant MIGRATIONS list.
  • Craftsmanship gate: layers, # source: on the new constants and the method size limit all hold.

Notes, no action before merge

  • CHANGELOG.md:16: the slice 1 entry still says "Persistence and extraction of the field are later slices", which contradicts the slice 2 entry just below it.
  • test_claim_event_time_persistence.py:63: rollback() does nothing on PG because the connection is autocommit. The PG variants leave orphan wiki.claim_events rows behind, since that table is not in _TABLES_TO_CLEAN.
  • The DDL defaults (pg_schema.py:194, sqlite_schema_wiki.py:34) are never exercised on a fresh store, because insert_claim_events always sends the Python default. A raw INSERT without event_time would pin them.

Verdict: approve

@github-actions
github-actions Bot merged commit d377989 into main Nov 6, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants