looks great! will review it sunday or monday.

looks good, I tried various custom mutators with python and without if with this env or not it creates crashes with an asan build and it did not.

is there a paper about this grammar mutator? if so please send a PR that adds a link to the readme.

merging now :)

Thank you for merging the PR so quickly! :)

A paper is currently under review, but I will come back upon acceptance (hopefully). Maybe a pre-print might be useful. Let's see.

We also work on a follow-up project with ATNwalk. So expect more stuff to come in the future ;)

looking forward to that :)

do you have a fuzzbench fuzzer setup for this?

Unfortunately, I don't have a fuzzbench setup for this. I am also not sure whether it is easily (read: quickly) doable. I considered it in the past, but then dropped it due to the time constraints I had in the project. There was neither a setup for Nautilus nor for Gramatron that I could've simply adapt. In general, I couldn't find something nice for grammar-based fuzzers. But maybe I am wrong and just couldn't find anything?

Hence, I've written my own testbed which did the job for me. I am not against having something "more official", but, at least right now, I am not willing to spend the additional effort when the testbed covers my use cases, in particular: benchmarking and developing :)

There is a nautilus setup based on libafl on fuzzbench. Gramatron custom mutator is super easy to set up. Would only leave atn, I would need to check how complex that is to set up.
but for bugs only xml and php benchmarks exist, so it is a limited test.
I could take care of that this week

If you don't mind and want to set it up, go ahead, I'd look forward to that :)

However, I don't have a grammar for XML.

The procedure to setup ATNwalk is pretty straight forward:

• Dependencies are in the Dockerfile of the testbed.
• Grammars are in the grammars repo

In fact, if you want to shortcut it, it should be no more than adding the install scripts from the testbed into the Dockerfile and run them!

I.e., append the following line to the Dockerfile mentioned above and it will install AFL++, the mutator, ATNwalk, and will also include the above grammars:

RUN git clone https://github.com/atnwalk/testbed && bash /home/rocky/testbed/install/atnwalk.bash

Running ATNwalk it is described here, but basically is no more than:

# create the required a random seed first
mkdir -p ~/campaign/example/seeds
cd ~/campaign/example/seeds
head -c1 /dev/urandom | ~/atnwalk/build/javascript/bin/decode -wb > seed.decoded 2> seed.encoded

# create the required atnwalk directory and copy the seed
cd ../
mkdir -p atnwalk/in
cp ./seeds/seed.encoded atnwalk/in/seed
cd atnwalk

# assign to a single core when benchmarking it, change the CPU number as required
CPU_ID=0

# start the ATNwalk server
nohup taskset -c ${CPU_ID} ${HOME}/atnwalk/build/javascript/bin/server 100 > server.log 2>&1 &

# start AFL++ with ATNwalk
AFL_SKIP_CPUFREQ=1 \
  AFL_DISABLE_TRIM=1 \
  AFL_CUSTOM_MUTATOR_ONLY=1 \
  AFL_CUSTOM_MUTATOR_LIBRARY=${HOME}/AFLplusplus/custom_mutators/atnwalk/atnwalk.so \
  AFL_POST_PROCESS_KEEP_ORIGINAL=1 \
  ~/AFLplusplus/afl-fuzz -t 100 -i in/ -o out -b ${CPU_ID} -- ~/jerryscript/build/bin/jerry

@voidptr127 could you email me your submitted paper? Or the results of your testbed runs? vh(at)thc(dot)org

I started to make fuzzers for fuzzbench for nautilus, gramatron and autotoken, but then for atnwalk you use rocky instead of ubuntu which complicates porting that to fuzzbench etc. and there would only be one target for benchmarking - so I think it is more of a time waste than helpful what I did so far.
magma has sqlite, xml, lua and php, so it does not fit much better.