Skip to content
This repository was archived by the owner on Apr 8, 2025. It is now read-only.
This repository was archived by the owner on Apr 8, 2025. It is now read-only.

QACandidates with wrong indices for a no_answer prediction #729

@brandenchan

Description

@brandenchan

When using FARM via Haystack for open domain QA, I am often getting the following ERROR

ERROR - farm.modeling.predictions -   Both start and end offsets should be 0: 1053, 1053 with a no_answer. 

On closer inspection, it seems not all our QACandidates are being converted to document level. I suspect the issue lies in aggregate_preds( )

preds_d = self.aggregate_preds(preds, passage_start_t, ids, seq_2_start_t)

This means that in our list of final predictions, we are getting a mix of no_answer QACandidates: some with QACandidate.aggregation_level = 'passage' where start = end = 1053 for example and some with QACandidate.aggregation_level = 'document' where start = end = 0.

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions