Skip to content

(Willing to PR) Make tokenizer.padding_side an argument instead of only being a field #30447

Description

@fzyzcjy

Feature request

Hi thanks for the library! When using tokenizer, for example, for batch-generation with GPT2 (in https://discuss.huggingface.co/t/batch-generation-with-gpt2/1517), it seems that currently I have to do something like:

tokenizer.padding_side = 'left'
data = tokenizer(['sentence one', 'another'])
tokenizer.padding_side = 'right'

Therefore, it would be great to have:

data = tokenizer(['sentence one', 'another'], padding_side = 'left')

just like what we do today for many options like padding_strategy etc.

Motivation

(see above)

Your contribution

Yes, I am willing to PR

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions