Skip to content
This repository was archived by the owner on Apr 8, 2025. It is now read-only.

Cross-validation for Question-Answering#335

Merged
Timoeller merged 20 commits intomasterfrom
cross_validate_qa
Apr 25, 2020
Merged

Cross-validation for Question-Answering#335
Timoeller merged 20 commits intomasterfrom
cross_validate_qa

Conversation

@bogdankostic
Copy link
Copy Markdown
Contributor

@bogdankostic bogdankostic commented Apr 22, 2020

This PR adds cross validation for Question Answering.
It is especially useful for evaluating small QA Domain datasets with long documents.

Features:

  • Optimally balanced document wise splitting of folds (realistic splits)
  • Down sampling of negative examples in train set (useful for long documents)
  • New recall @ n metric that uses multiple predictions

@bogdankostic bogdankostic requested a review from Timoeller April 22, 2020 18:21
Copy link
Copy Markdown
Contributor

@Timoeller Timoeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the test for tensor indices.
Apart from that it looks very good.

@bogdankostic bogdankostic changed the title WIP Cross-validation for Question-Answering Cross-validation for Question-Answering Apr 24, 2020
@Timoeller Timoeller merged commit 5398d6f into master Apr 25, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants