Hi,
I'd like to run finetune.py with wmt dataset in TPU to train from scratch not finetune. I appreciate the response to some questions:

1. Why there are two versions of fine-tuning using Seq2SeqTrainer and finetune.py and which one is suitable for my usecase?
2. Seq2SeqTrainer does not support predict on TPU, is this the case for finetune.py as well?
3. when running on TPU, it is written to use xla_spawn.py, since the codes of finetune.py are with pytorch lightening, is this needed to launch them with xmp.spawn(mod._mp_fn, args=(), nprocs=args.num_cores)?
4. in the finetune.py dataloader, I see this is distributed based on number of gpus, but I cannot see this is distributed also when one uses tpus, Is this taking care of making dataloader distributed in case of using TPU automatically?
5. If using finetune.py with TPU, is there any specific setup I need to add for finetuning/training T5 on WMT?
6. I assume one need to use maybe something like this as a sampler for TPU distributed dataloaders, I see this is not the case in the codes of finetune.py, does data parallelism works in finetune.py?
7. when to use sortish_sampler/dynamic_sampler, do they work on TPUs?

  sampler = torch.utils.data.distributed.DistributedSampler(
        dataset,
        num_replicas=xm.xrt_world_size(),
        rank=xm.get_ordinal(),
        shuffle=True)

8. fp16 does it work with TPUs as well?
9. with iterable datasets seems dataloader in finetune_trainer is not working with distributed detup on TPU, do you know how to implement it? am I missing something? thanks
   thank you very much.