hi @akhikolla there is a problem in your current implementation.

Right now deepstate_compile_fun runs the test harness with multiple inputs under valgrind. This is problematic for several reasons

1. sometimes valgrind aborts without saving fun_output/.pass and fail binary files. e.g. rcpp_read_out_of_bound: "new/new[] failed and should throw an exception, but cannot throw exceptions and so is aborting instead. Sorry." So valgrind aborts before deepstate has a chance to save the pass/fail binary files. In contrast if you run the test harness without valgrind then these pass/fail binary files are saved to fun_output/
2. it is very slow so there are not many random inputs that can be tested.
3. the valgrind log file is useless because it contains messages from all of the different random inputs.

To fix I propose that you change the code to

1. deepstate_compile_fun: run initial test harness NOT under valgrind, don't save any logs, just save fun_output/*.pass or fail files.
2. deepstate_analyze_fun: run deepstate_analyze_fun_inputs (new function, see below) on all pass/fail binary files.
3. deepstate_analyze_fun_inputs("path/to/pkg/inst/testfiles/fun_output/HASH.fail") run test harness with this set of inputs under valgrind, save and return valgrind messages and R inputs.

what do you think? does that make sense? can you code that for next week please?