from transformers import *
args = PyTorchBenchmarkArguments(
models=["gpt2_pt"],
batch_sizes=[1], sequence_lengths=[8])
config_base = GPT2Config(n_layer=2, n_head=2)
config_base.architectures = ["GPT2LMHeadModel"]
benchmark = PyTorchBenchmark(args, configs=[config_base])
print(benchmark.run())
args = TensorFlowBenchmarkArguments(
models=["gpt2_tf"],
batch_sizes=[1], sequence_lengths=[8])
config_base = GPT2Config(n_layer=2, n_head=2)
config_base.architectures = ["GPT2LMHeadModel"]
benchmark = TensorFlowBenchmark(args, configs=[config_base])
print(benchmark.run())
Without LMHeadModel, the benchmark is 6ms vs 8ms.
The benchmark shows the Tensorflow TFGPT2LMHeadModel is 7 times (140ms) slower than Torch GPT2LMHeadModel implementation (20ms). Based on the profile tool from latest https://www.tensorflow.org/tfx/serving/tensorboard, the bottleneck is here.
transformers/src/transformers/modeling_tf_utils.py
Line 796 in d9149f0
Without LMHeadModel, the benchmark is 6ms vs 8ms.