@timogoebel Can you elaborate on the downsides ?

@leewaa: I don't like it because it's odd to use the relative path to go back to the "root-path". I guess I just don't like the developer experience.
I'm wondering if it's possible to just extend the include path of ruby to make it find the test_helper.

If I run be bin/rake test:foreman_dlm, it also explicitly sets the include path:

/ruby -I"lib:test:/Users/f0218175/foremandev/foreman_dlm/lib/tasks/../../test" -I"/Users/f0218175/.rvm/gems/ruby-2.5.0/gems/rake-12.3.1/lib" "/Users/f0218175/.rvm/gems/ruby-2.5.0/gems/rake-12.3.1/lib/rake/rake_test_loader.rb"

My gut feeling tells me, the proper approach is to try to do the same. But I'd need to dig into this to know for sure.

Ok I see.

Well I did a hunt around to be totally sure its not going to cause us any problems (still can't know for sure). Checkout this thread on Stack overflow. Its usage is not clearly defined however some seem to think that require_relative is for requiring files local to the project, and require for files in external libraries, but that is still disputed I think.