Nice idea, both for this test/bench and in general, i.e. I'm also in favour of having a dedicated bench(mark) directory apart from tests.

However, IMHO the output should be adapted to be (machine) readable, e.g. CSV with columns like:

[edit/add]: or some JSON style data format to allow for meta infos to, like number of runs, test name etc.

IMHO it does not really need to be directly machine readable (as in JSON or CSV), just a common format used for different applications would suffices. I would expect the app to be fired from some kind of pexpect script or similar in anycase, and the conversion from simly applying regexes to JSON/CSV/whatever can be done in python then...

I agree that JSON might be a bit overkill, but a simple CSV format will make it parsing/conversion using python easier, too.

At least the current output is a bit chatty for my taste (ie., having the complete runtime and per call, does not provide any additional info). Further, every unnecessary char in output strings consumes precious memory on very constraint platforms such as Arduino (i.e., uno, duemilanove).

At least the current output is a bit chatty for my taste (ie., having the complete runtime and per call, does not provide any additional info). Further, every unnecessary char in output strings consumes precious memory on very constraint platforms such as Arduino (i.e., uno, duemilanove).

I think a bit chatty output is actually helpful, as I see this app not only for automated test runs, but also for manual runs while optimizing code. And I think the parsing part is not an issue here: the regex to parse the proposed output in python is dead simple...

Memory wise I also think the difference is not that much and the applications itself are quite simple and therefore dont need much memory in the first place, so that the advantage of nicely readable lines prevails here.

I agree that JSON might be a bit overkill, but a simple CSV format will make it parsing/conversion using python easier, too.

+1

Maybe we write a central function, like:

void bench_put_result(const char *benchmark, const char *metric, const char *value)

... and centrally write in whatever format works best?

I am however not sure, if this kind of application should reside in the tests/ folder. I tend to think that a dedicated bench/ folder would make sense here. Any opinions on this?

IMO "test" can be used for "benchmark test", so for now, IMO we can stick to tests/. Prefixing with "bench_" goes a long way.

I just thought about rowing back with the dedicated benchmark application and rather extend the gpio test application with a bench shell command (as done some time ago also for the periph_spi test application). Automatic execution of the benchmark is still possible using a Pexpect script. And it would mean that the benchmark is run on a variable device definition (-> GPIO_PIN(x,y) is set at runtime), which further prevents the compiler to do certain optimizations which it could do on a constant pin definition...

Opened a new PR in #8040 implementing the same code as shell command in the existing test. I like it much better, so if no one objects, I'd like to close this PR in favor of #8040.

I'd also prefer to go with #8040 and close this one. @smlng d'accord?

As I see it: three maintainers for #8040 over #7478 -> reason enough to close this one