System Info
Name: optimum
Version: 1.8.4
Summary: Optimum Library is an extension of the Hugging Face Transformers library, providing a framework to integrate third-party libraries from Hardware Partners and interface with their specific functionality.
Home-page: https://github.com/huggingface/optimum
Author: HuggingFace Inc. Special Ops Team
Author-email: [email protected]
License: Apache
Location: /home/jiawei/.conda/envs/g/lib/python3.8/site-packages
Requires: coloredlogs, datasets, huggingface-hub, numpy, packaging, sympy, torch, torchvision, transformers
Required-by:
Ubuntu 20.04; Python 3.8; 4xA6000 GPUs
Who can help?
@fx
Information
Tasks
Reproduction
from transformers import AutoTokenizer, AutoModelForCausalLM
from optimum.bettertransformer import BetterTransformer
tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen2-16B")
model = AutoModelForCausalLM.from_pretrained(
"Salesforce/codegen2-16B", trust_remote_code=True, revision="main"
)
model = BetterTransformer.transform(model, keep_original_model=True)
text = "def hello_world():"
input_ids = tokenizer(text, return_tensors="pt").input_ids
generated_ids = model.generate(input_ids, max_length=512)
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))Expected behavior
It should work as this is basically the codegen architecture which is stated to be supported in https://huggingface.co/docs/optimum/v1.8.3/bettertransformer/overview
However, it will fail by throwing an exception saying None has no attribute of ...
System Info
Who can help?
@fx
Information
Tasks
examplesfolder (such as GLUE/SQuAD, ...)Reproduction
Expected behavior
It should work as this is basically the codegen architecture which is stated to be supported in https://huggingface.co/docs/optimum/v1.8.3/bettertransformer/overview
However, it will fail by throwing an exception saying
Nonehas no attribute of ...