Skip to content

Abstention: add an entity-existence gate before retrieval (as in the tangleclaw-style row) #114

Open

Description

@tonydzi

The tangleclaw-style reference row (thanks to Jason Vaughan, #87) abstains on 10/10 absent_entity questions with a cheap capitalized-entity check, and refuses only 3 of 69 answerable questions. Our vector row abstains on 6/10 absent entities and refuses 18/69 answerable ones.

Proposal: run an entity-existence check before retrieval: if a capitalized name in the question matches no note title, alias or entity in the index, return no answer. It needs no model and should lift absent_entity toward 1.00 while letting the reranker threshold be relaxed (see the paraphrase issue). Acceptance: abstain_correct on absent_entity 1.00, no new false abstains on fact or bridge.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions