You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[bug] wiki_refine_draft sets every refined draft's confidence to 0.85, lifting it past the curator's approve threshold #578
wiki_refine_draft (handler_refine, mcp_server/handlers/wiki_refine.py:300) overwrites the draft's confidence with a fixed 0.85 every time an LLM submits refined prose, and records the audit memo with the same 0.85 (line 315). No source backs the value: ADR-0467 preserves the file as it was during #514 and records no threshold choice.
The template synthesizer sets a draft's confidence to the mean confidence of its source claims, or 0.4 when no claim carries one (mcp_server/core/draft_synthesizer.py:271-277). The curator uses that number as a gate: below MIN_CONFIDENCE_APPROVE = 0.6 a draft gets a reason and can at best be held, below MIN_CONFIDENCE_HOLD = 0.4 it is rejected (mcp_server/core/draft_curator.py:15-16, 99-110, 138-141). wiki_curate then moves approved drafts toward wiki_compile.
So rewriting a draft's prose lifts it above the approve threshold whatever the evidence behind its claims. A draft synthesized at 0.4 from weak claims clears the confidence check of evaluate_draft once refined, although no claim changed.
Reproduction steps
Run wiki_extract then wiki_synthesize on any store; pick a pending draft and note its confidence.
Call wiki_refine_draft with that draft_id and a new lead.
Read the draft again: confidence is 0.85, and evaluate_draft no longer reports "below approve threshold" for it.
Expected behavior
Refinement changes the wording of a draft and leaves its confidence as the source claims set it. The audit memo records the draft's confidence as it stands, not a constant.
Evidence required to close
A test on the SQLite backend that refines a draft whose confidence is below MIN_CONFIDENCE_APPROVE and asserts that its confidence is unchanged, that the refined_llm memo carries that confidence, and that evaluate_draft still reports the approve-threshold reason.
Correction on the reach of this defect, checked after opening the issue.
wiki_get_draft and wiki_refine_draft are not registered as MCP tools: mcp_server/tool_registry_wiki.py registers ten wiki tools and neither of these, and git log -S wiki_refine over the tool_registry*.py files finds no commit that ever did. The only caller of wiki_refine in the tree is test_refine_lists_pending_drafts, which exercises handler_get.
In the maintainer's PostgreSQL store (read-only query, 2026-09-16), wiki.memos holds a single refined_llm memo: id 105, 2026-04-14, synth_modeltest_runner_v1, confidence 0.85. The 546 drafts all carry synth_modeltemplate_v1.
So the overwrite is latent: no draft reaches it through the MCP surface today. The fix still stands, because the handler is the documented Path B and whoever wires it would inherit the 0.85.
Activity
cdeust commentedon Sep 15, 2026
Correction on the reach of this defect, checked after opening the issue.
wiki_get_draftandwiki_refine_draftare not registered as MCP tools:mcp_server/tool_registry_wiki.pyregisters ten wiki tools and neither of these, andgit log -S wiki_refineover thetool_registry*.pyfiles finds no commit that ever did. The only caller ofwiki_refinein the tree istest_refine_lists_pending_drafts, which exerciseshandler_get.In the maintainer's PostgreSQL store (read-only query, 2026-09-16),
wiki.memosholds a singlerefined_llmmemo: id 105, 2026-04-14,synth_modeltest_runner_v1, confidence 0.85. The 546 drafts all carrysynth_modeltemplate_v1.So the overwrite is latent: no draft reaches it through the MCP surface today. The fix still stands, because the handler is the documented Path B and whoever wires it would inherit the 0.85.
fix(wiki): refining a draft leaves its confidence as the claims set it (