Test case generation for 3d #101
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
huge speedup: all positive ELF._E_IDENT witnesses now found within 3 s
Perf issue with bitwise not
HUGE performance issues
…aramana_3d_z3_testgen
with @nikswamy: it is no longer necessary to build the witness as a sequence in z3/SMT2
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request generates test cases for 3d data format specifications using z3.
The latest (valid) build for this PR contains a Linux standalone binary package for EverParse as a build artifact called
everparse, downloaded aseverparse.zip. Warning: this .zip will extract many files and directories to the current directory.Usage:
bash everparse.sh <options> <files.3d>...Supported scenarios
--z3_test MyModule.my_parserproduces positive and negative test cases formy_parserdefined inMyModule.3d--z3_diff_test MyModule1.my_parser1,MyModule2.my_parser2produces a set of test cases that are positive formy_parser1but negative formy_parser2, and a set of test cases for the converse--emit_smt_encodingproduces the SMT encoding corresponding to the .3d filesFurther options
--batchproduces, compiles and runs a test program,test.exe, to check whether the generated test cases are actually accepted or rejected by the verified validator implementations for the given 3d parser specifications. This option is set by default when using theeverparse.shscript from the binary package. Use--no_batchto disable it and only produce the test cases.--z3_witnesses 18produces 18 test cases.--z3_test_mode pos(resp.--z3_test_mode neg) only produces positive (resp. negative) test cases with--z3_test. (If this option is not used, then 3d will produce both positive and negative tests.)--z3_executable path/to/z3.exeto specify a path to the Z3 executable that should be used to generate the test cases. This option does not affect the verification of the generated F* files. Test cases are best generated with the latest z3 (4.12.2), but verification of the generated F* files works only with Z3 4.8.5 at the moment. The standalone binary package on Linux includes both versions of z3 andeverparse.shalready sets--z3_executableaccordingly.