Skip to content

Fix seqeval metric#810

Merged
sgugger merged 1 commit into
masterfrom
fix_seqeval_metric
Nov 9, 2020
Merged

Fix seqeval metric#810
sgugger merged 1 commit into
masterfrom
fix_seqeval_metric

Conversation

@sgugger

@sgugger sgugger commented Nov 6, 2020

Copy link
Copy Markdown
Contributor

The current seqeval metric returns the following error when computed:

~/.cache/huggingface/modules/datasets_modules/metrics/seqeval/78a944d83252b5a16c9a2e49f057f4c6e02f18cc03349257025a8c9aea6524d8/seqeval.py in _compute(self, predictions, references, suffix)
    102         scores = {}
    103         for type_name, score in report.items():
--> 104             scores[type_name]["precision"] = score["precision"]
    105             scores[type_name]["recall"] = score["recall"]
    106             scores[type_name]["f1"] = score["f1-score"]

KeyError: 'LOC'

This is because the current code basically tries to do:

scores = {}
scores["LOC"]["precision"] = some_value

which does not work in python. This PR fixes that while keeping the previous nested structure of results, with the same keys.

@lhoestq lhoestq left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch thanks !

@sgugger
sgugger merged commit 92acf1e into master Nov 9, 2020
@sgugger
sgugger deleted the fix_seqeval_metric branch November 9, 2020 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants