Skip to content

Allow to run some tests isolated #2142

@pbludov

Description

@pbludov

We have one test that changes the global state, for example

@Test
void localeTest() {
  Locale.setDefault(Locale.FRENCH);
  assertEquals("Bunjour", getMessage());
}

and hundreds of tests that reads the state:

@Test
void someTest() {
  assertEquals("Hello", getMessage());
}

After enabling parallel test execution our tests start to fail at random locations with messages like that:
[ERROR] message 0 expected:<...Annotation.java:17: [Annotation 'AnnotationAnnotation' have incorrect indentation level 2, expected level should be 0].> but was:<...Annotation.java:17: [Die Annotation 'AnnotationAnnotation' hat eine unerwartete Einrückungstiefe von 2 (erwartet: 0)].>

@Execution(ExecutionMode.SAME_THREAD) for the test localeTest won't help us, since the all other tests are able to run concurrent with every other test and with localeTest too.

This can be solved with @ResourceLock:

@Test
@Execution(ExecutionMode.SAME_THREAD)
@ResourceLock(value="global", mode=READ_WRITE)
void localeTest() {
  Locale.setDefault(Locale.FRENCH);
  assertEquals("Bunjour", getMessage());
}

@Test
@Execution(ExecutionMode.CONCURRENT)
@ResourceLock(value="global", mode=READ)
void test1() {
}

// ...

@Test
@Execution(ExecutionMode.CONCURRENT)
@ResourceLock(value="global", mode=READ)
void test100500() {
}

but this is error prone, since adding a new Test without this annotation will produce a flaky tests which may fail.

It will be much better to mark some tests as "incompatible with parallel execution at all":

@Test
@Execution(ExecutionMode.ALONE) // no other this will be executed in parallel 
void localeTest() {
 Locale.setDefault(Locale.FRENCH);
 assertEquals("Bunjour", getMessage());
}

As a workaround, it is possible to turn such a test into Junit 4 test, which will be executed in other universe, isolated from all other.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions