Skip to content

health: flag a source that calls itself unverified but has no source_quality (#258) - #377

Merged
eugeniughelbur merged 2 commits into
eugeniughelbur:mainfrom
tonydzi:feat/source-quality-prose-label
Oct 28, 2026
Merged

eugeniughelbur merged 2 commits into
eugeniughelbur:mainfrom
tonydzi:feat/source-quality-prose-label

Conversation

@tonydzi

@tonydzi tonydzi commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Mycroft here, Anton's synthetic AI co-founder, opening this for @tonydzi.

Refs #258: the piece of the next slice the maintainer accepted on 24.10, a health warning for a source whose body says it is speculative while its frontmatter says nothing. The compile-time gate in /obsidian-ingest step 6 and /obsidian-synthesize is not in this PR.

What changes

scripts/vault_health.py gets check_source_quality_unlabelled, registered as Source quality (unlabelled) next to Source quality. It reads type: source notes with no source_quality key at all, takes the first 12 non-empty body lines, strips code from that window, and warns when it finds one of unverified, not verified, unconfirmed, speculative, speculation, provenance unknown / provenance: unknown. The message quotes the matched phrase. "source_quality_policy": "silent" mutes it, same contract as the existing check.

Deliberate choices:

  • summary and not verbatim are not markers, because references/source-quality-check.md says they do not by themselves lower the grade.
  • A present but malformed label (typo, empty, a YAML list) is skipped. That note needs a different fix than "add the field", and the message would be wrong advice.
  • The window is cut before code is stripped, so a long code block at the top uses up the window instead of pulling deeper prose into it.

Docs: CHANGELOG.md (Unreleased), commands/obsidian-health.md, references/source-quality-check.md (Consumers).

Tests

tests/test_source_quality_unlabelled.py, 21 tests, synthetic fixtures only. Negative cases pinned hardest: labelled sources, sources that never use the words, untitled / unknown alone, the word in a fenced block, in inline code, after a 30-line code block, 40 lines deep, a non-source note, the silent policy, and a malformed label.

Red proof: against the first draft of the check, 5 of these fail (the colon form, the three malformed labels, the long code block). Against a mutant that ignores the label, test_a_labelled_source_is_silent fails; against one that scans the whole body, the code-block and deep-capture tests fail.

pytest -q tests/test_source_quality_unlabelled.py tests/test_source_quality.py   # 41 passed
pytest -q --deselect tests/test_p1_p2_fixes.py::test_ordinary_public_urls_still_work   # 1232 passed, 1 skipped, 1 xfailed
ruff check .                                   # All checks passed
python scripts/sweep_non_ascii.py --check      # CHECK PASSED
python scripts/conformance_report.py --check   # conformance board is current
python scripts/build_site.py --check           # Site is current

The deselected test resolves example.com, which the network on this machine does not; it is unrelated to this change.

🤖 Generated with Claude Code

Anton Dziatkovskii and others added 2 commits October 27, 2026 23:00
…quality (eugeniughelbur#258)

A correction that lives only in prose is invisible to every check that reads
fields. Source quality (unlabelled) reads the first 12 non-empty body lines of
each type: source note without a recognised grade, code removed, against a short
fixed marker list, and quotes the phrase it matched. Same silent policy key.

Assisted-by: Claude Code/claude-opus-5-5
Machine: MacPro192
Account: tonydzi
Operator: Anton
Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ugeniughelbur#258)

Review findings: a present but malformed source_quality was reported as
missing; "Provenance: unknown" was not matched; a long code block at the top
pulled deeper prose into the preamble window. Tests pin each, plus inline code
and lookalike words (untitled, unknown alone).

Assisted-by: Claude Code/claude-opus-5-5
Machine: MacPro192
Account: tonydzi
Operator: Anton
Co-Authored-By: Claude Opus 5.5 <[email protected]>
@tonydzi
tonydzi force-pushed the feat/source-quality-prose-label branch from 9df3d0c to 4908837 Compare October 27, 2026 23:01
@eugeniughelbur

Copy link
Copy Markdown
Owner

@tonydzi thanks, this is the health-warning piece of #258 I said yes to.

  • The check only reads type: source notes with no source_quality key, so labelled vaults stay quiet.
  • Cutting the window before stripping code is the right order, and the tests pin it.
  • I ran the same matching over my own vault's source notes: zero warnings, so it adds no noise to an existing vault.
  • CI is green on Linux and Windows.

Merging.

One open thought, not a blocker: a source about "speculative decoding" or market "speculation" would match on topic, not on provenance. If that shows up in real use, a narrower phrase list is the fix.

@eugeniughelbur
eugeniughelbur merged commit dcdcbc3 into eugeniughelbur:main Oct 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants