Describe the bug
I am training a model for aspect-based sentiment analysis (trying to extract the sentiment of specific aspects of the text). To achieve this goal, I am using the Text-Pair Classification method of FARM. The model is training fine, but when I try to apply inference, I get wrong and static result independent of my text_b input. This suggests to me that the inference is not using 'text_b' correctly.

Error message
Here there are some examples:

Expected behavior
If not outputting the correct label, the final probability should at least be somewhat sensitive to what I input as "text_b".

Additional context
I tested if the training was considering this a classification task instead of a text-pair classification task. I substituted TextPairClassificationProcessor for TextClassificationProcessor and, sure enough, the accuracy decreased substantially. So it is not like "text_b" is being ignored on training time.

System:

• OS: Google Colab
• GPU/CPU: K80
• FARM version: 0.7.0