Again, you have multiple assertThat in each test. You should split tests even more. You see, each scenario is fully independent from the rest. You should also remove the temporary variables.

Test names should describe the scenario so that you can remove the comments completely. A good code is usually understandable without comments. You should be able to make the code self-documenting rather than resorting to a comment. Comments that document the "what" are usually easy to get rid of; it's the "why" comments that are hard to replace. Use a comment when it is infeasible to make your code self-explanatory.

@Test
public void noUrlNoInput_long_isNotValid() {
   assertThat(validateArguments(new String[]{
      "--output_base", "output value",
      "--threads", "4"
    })).isFalse();
}

Also, there is no difference between scenarios with (--output_base, --threads) vs (-o, -t) since url and input are not provided anyway. You may have instead:

@Test
public void noUrlNoInput_isNotValid() {
   assertThat(validateArguments(new String[]{})).isFalse();
}

Thanks for the heads-up @aababilov!