Describe the bug
Tokenizer becomes very slow with large custom vocab.

Additional context
This was introduced after switching to the Tokenizers from the transformers repo

There are related issues reported in the transformers repo:

• GPT2 tokenizer is so slow because of sum() huggingface/transformers#1830
• tokenization slow huggingface/transformers#1621
• convert list to set in tokenize().split_on_tokens() huggingface/transformers#1881

To Reproduce

• Add custom vocab to tokenizer via tokenizer.add_tokens()
• Load some data into the data silo, e.g. run examples/lm_finetuning.py

System:

• OS: Ubuntu 18.04
• GPU/CPU: Both
• FARM version: master @ 484d26c