Description:

LongformerAttention might need >2GB memory for large batch size. GetScratch1Size function might encounter integer overflow. For example batch_size=64, sequence_length=4096, window=256, num_heads=12, element_size=2. The value will be larger than 2GB thus cause integer overflow. If user uses smaller batch size like 16 (or 32), this error will not be triggered.

Motivation and Context

• Why is this change required? What problem does it solve?
• If it fixes an open issue, please link to the issue here.

LongformerAttention cuda might fail with large batch size. For debug build, assertion error will be triggered at assert(current_pointer == scratch2). For release build, the error is like onnxruntime::CudaCall] CUDA failure 700: an illegal memory access was encountered