I just download mTet-v2 via an HF dataset link provided in this comment which contains 4.19M training pairs. However, I find that 9.9K mTet-v2 training pairs appear in the PhoMT test set.
Please assist in correcting this mTet-v2 dataset for a proper evaluation. Thank you!
P.S.: You can find the English training sentences in mTet-v2 that appear in the PhoMT test set in the attached file: mtet-training-pairs-in-phomt-test-set.en.txt
I just download mTet-v2 via an HF dataset link provided in this comment which contains 4.19M training pairs. However, I find that 9.9K mTet-v2 training pairs appear in the PhoMT test set.
Please assist in correcting this mTet-v2 dataset for a proper evaluation. Thank you!
P.S.: You can find the English training sentences in mTet-v2 that appear in the PhoMT test set in the attached file: mtet-training-pairs-in-phomt-test-set.en.txt