Skip to content
This repository was archived by the owner on Apr 8, 2025. It is now read-only.
This repository was archived by the owner on Apr 8, 2025. It is now read-only.

Trainer does not use loaded model for test set evaluation #522

Closed
Closed
Trainer does not use loaded model for test set evaluation#522
@PhilipMay

Description

@PhilipMay
PhilipMay
opened on Sep 6, 2020

When using early stopping the best models are saved. Later they are loaded here:

FARM/farm/train.py

Lines 355 to 356 in c6871b6

model = AdaptiveModel.load(self.early_stopping.save_dir, self.device, lm_name=lm_name)
model.connect_heads_with_processor(self.data_silo.processor.tasks, require_labels=True)

But here the evaluator does not get the loaded model model but the old model self.model:

FARM/farm/train.py

Line 365 in c6871b6

self.test_result = evaluator_test.eval(self.model)

This is a bug...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions