This was started as two separate GitHub issues, but then I thought I should merge them. — The idea is:
clarinet test can support property tests, and those tests may be written in either TypeScript or Clarity
clarinet fuzz can then turn those tests into fuzz tests
Context
Clarinet tests are essentially Deno tests. This has the interesting side-effect of being able to use what's already available in JS/TS ecosystem when testing Clarity code. (For example, fast-check and dspec can be used, as we've done here with @LNow.)
A typical workflow is declaring functions in Clarity and then writing tests in TypeScript. Clarinet can then discover those tests (with the help of Deno) and run them. It would be practical however to also discover tests written in Clarity and run those as well.
Discoverability
In addition to existing test suite(s) in TypeScript, Clarinet can execute as a test any Clarity function that meets the following criteria:
- The method is public
- The method name starts with
test (configurable in Clarinet.toml)
(If a public function named beforeEach is present it can be executed before each test is run. Such a function can exist in the style of testing @jcnelson does here, may be discussed also on a separate thread, however.)
| type |
prefix |
arguments |
semantics |
| concrete |
test |
no |
single execution with concrete values |
| property |
test |
yes |
multiple executions with randomly generated concrete values (smaller values) |
| fuzz |
test |
yes |
multiple executions with randomly generated concrete values (biased)* |
*Also configurable, in Clarinet.toml.
Fuzz mode can be enabled via new clarinet fuzz command. Essentially it can work the same as clarinet test but use a different fast-check configuration.
Libraries
Both test suites (TypeScript, and Clarity (hypothetically)) can be discovered and run from the context of Deno, and in the case of property tests and fuzz tests, the generated data can be provided by fast-check.
I have been using (and getting an inside look into) fast-check recently. It has all the modern features of a prop/fuzz testing library, e.g. model testing, integrated shrinking, control over the scope of generated values, and many other useful functions.
Pros and cons
What are the advantages and disadvantages of having the option to write tests in Clarity?
- One disadvantage that @LNow pointed out to me is that this can screw the execution costs.
- An advantage can be that there's less context switching; you write your contract in Clarity and your tests in Clarity as well. (No need to marshal stuff between different languages (as done here for example).
- The possibility to encode (some of the) smart contract audits in code that (continuously) runs and checks those tests, instead of having all the audits based on a specific git commit and PDF file.
- (more pros and cons to be added)...
This was started as two separate GitHub issues, but then I thought I should merge them. — The idea is:
clarinet testcan support property tests, and those tests may be written in either TypeScript or Clarityclarinet fuzzcan then turn those tests into fuzz testsContext
Clarinet tests are essentially Deno tests. This has the interesting side-effect of being able to use what's already available in JS/TS ecosystem when testing Clarity code. (For example, fast-check and dspec can be used, as we've done here with @LNow.)
A typical workflow is declaring functions in Clarity and then writing tests in TypeScript. Clarinet can then discover those tests (with the help of Deno) and run them. It would be practical however to also discover tests written in Clarity and run those as well.
Discoverability
In addition to existing test suite(s) in TypeScript, Clarinet can execute as a test any Clarity function that meets the following criteria:
test(configurable inClarinet.toml)(If a public function named
beforeEachis present it can be executed before each test is run. Such a function can exist in the style of testing @jcnelson does here, may be discussed also on a separate thread, however.)testtesttest*Also configurable, in
Clarinet.toml.Fuzz mode can be enabled via new
clarinet fuzzcommand. Essentially it can work the same asclarinet testbut use a different fast-check configuration.Libraries
Both test suites (TypeScript, and Clarity (hypothetically)) can be discovered and run from the context of Deno, and in the case of property tests and fuzz tests, the generated data can be provided by fast-check.
I have been using (and getting an inside look into) fast-check recently. It has all the modern features of a prop/fuzz testing library, e.g. model testing, integrated shrinking, control over the scope of generated values, and many other useful functions.
Pros and cons
What are the advantages and disadvantages of having the option to write tests in Clarity?