System Info

• transformers version: 4.36.2
• Platform: macOS-13.4-arm64-arm-64bit
• Python version: 3.10.10
• Huggingface_hub version: 0.20.2
• Safetensors version: 0.4.0
• Accelerate version: 0.23.0
• Accelerate config: not found
• PyTorch version (GPU?): 2.1.0 (False)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: no
• Using distributed or parallel set-up in script?: no

Who can help?

@ArthurZucker @younesbelkada

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

export TASK_NAME=mrpc

python examples/pytorch/text-classification/run_glue.py \
  --model_name_or_path bert-base-cased \
  --task_name $TASK_NAME \
  --do_train \
  --do_eval \
  --max_seq_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --output_dir /tmp/$TASK_NAME/

Expected behavior

Issue Description:
The run_classification.py script currently has an issue where the assignment of id2label in the model's config is incorrect. The problem arises from copying config.label2id without modifying it later on. This issue was introduced with transformers version 4.36.

Steps to Reproduce:

1. Execute the run_classification.py script with a configuration file.
2. Inspect the id2label attribute in the model's config.

Expected Behavior:
The id2label attribute should be assigned correctly, reflecting the label-to-id mapping.

Actual Behavior:
The id2label attribute is assigned based on the original config.label2id, leading to incorrect mapping.

Proposed Solution:
Modify the following line in run_classification.py:

model.config.id2label = {id: label for label, id in config.label2id.items()}

to:

model.config.id2label = {id: label for label, id in label_to_id.items()}

This change ensures that the correct mapping is used.