Skip to content
This repository was archived by the owner on Apr 8, 2025. It is now read-only.
This repository was archived by the owner on Apr 8, 2025. It is now read-only.

Add no_answer scores to QA evaluation #686

@Timoeller

Description

@Timoeller

When using FARMs Evaluator for QA evaluation we only have EM and f1 across all input examples.

The official SQuAD eval script has both, an overall score and at the same time scores for text answers vs no_answer.
We should also have both inside the Evaluator Class.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions