Skip to content

Display assertion failure immediately once falsified in text mode#1271

Merged
gustavo-grieco merged 1 commit intomasterfrom
showTxsInTextMode
Jun 13, 2024
Merged

Display assertion failure immediately once falsified in text mode#1271
gustavo-grieco merged 1 commit intomasterfrom
showTxsInTextMode

Conversation

@samalws-tob
Copy link
Copy Markdown
Collaborator

@samalws-tob samalws-tob commented Jun 12, 2024

Fixes #1205

Example:
Before:

Analyzing contract: /Users/sam/Documents/echidna-june-2024/echidna/contract.sol:Foo
[2024-06-12 11:23:17.57] Running slither on contract.sol... Done! (0.702374s)
[2024-06-12 11:23:18.27] [Worker 0] Test echidna_asdf falsified!
[2024-06-12 11:23:18.33] [Worker 0] New coverage: 256 instr, 1 contracts, 1 seqs in corpus
[2024-06-12 11:23:20.83] [Worker 0] Test limit reached. Stopping.
[2024-06-12 11:23:20.83] [status] tests: 1/1, fuzzing: 101/50000, values: [], cov: 256, corpus: 1
echidna_asdf: failed!💥  
  Call sequence:
    Foo.asdf(1)
    Foo.bsdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)

Traces: 



Unique instructions: 256
Unique codehashes: 1
Corpus size: 1
Seed: 3826264006055040584

After:

Analyzing contract: /Users/sam/Documents/echidna-june-2024/echidna/contract.sol:Foo
[2024-06-12 11:23:17.57] Running slither on contract.sol... Done! (0.702374s)
[2024-06-12 11:23:18.27] [Worker 0] Test echidna_asdf falsified!
  Call sequence:
Foo.csdf(57947087071538287116229096099518778219345908371936822082628725600138693436524) from: 0x0000000000000000000000000000000000020000 Time delay: 390247 seconds Block delay: 53678
Foo.csdf(115792089237316195423570985008687907853269984665640564039457584007913129639930) from: 0x0000000000000000000000000000000000020000 Time delay: 344203 seconds Block delay: 59552
Foo.bsdf(1) from: 0x0000000000000000000000000000000000030000 Time delay: 33605 seconds Block delay: 5053
Foo.csdf(0) from: 0x0000000000000000000000000000000000020000 Time delay: 117472 seconds Block delay: 1362
Foo.bsdf(1) from: 0x0000000000000000000000000000000000030000 Time delay: 332369 seconds Block delay: 11942
Foo.asdf(4370000) from: 0x0000000000000000000000000000000000020000 Time delay: 590349 seconds Block delay: 60267
Foo.asdf(3) from: 0x0000000000000000000000000000000000020000 Time delay: 113273 seconds Block delay: 60364
Foo.csdf(105328709578986275286875549616058049166248807697387859268196766548365176826234) from: 0x0000000000000000000000000000000000010000 Time delay: 45142 seconds Block delay: 50499
Foo.csdf(336) from: 0x0000000000000000000000000000000000010000 Time delay: 198598 seconds Block delay: 1984
Foo.bsdf(24495534881220516422514047452236933939549535757623267174267198127854015088056) from: 0x0000000000000000000000000000000000020000 Time delay: 522178 seconds Block delay: 37767
Foo.asdf(973821506116829516598871744363094460124616298305013842383472628649812407579) from: 0x0000000000000000000000000000000000030000 Time delay: 32767 seconds Block delay: 5054
Foo.asdf(62737908824103347051668683296154850097080317044880093722682446698363781686349) from: 0x0000000000000000000000000000000000030000 Time delay: 275394 seconds Block delay: 561
Foo.csdf(2) from: 0x0000000000000000000000000000000000030000 Time delay: 404997 seconds Block delay: 32737
Foo.bsdf(16673238516546302010005614697145054190922750209626982441491245815592674170811) from: 0x0000000000000000000000000000000000010000 Time delay: 73040 seconds Block delay: 23978
Foo.bsdf(0) from: 0x0000000000000000000000000000000000010000 Time delay: 400981 seconds Block delay: 5023
Foo.asdf(4369999) from: 0x0000000000000000000000000000000000020000 Time delay: 67960 seconds Block delay: 5140
Foo.bsdf(115792089237316195423570985008687907853269984665640564039457584007913129639929) from: 0x0000000000000000000000000000000000030000 Time delay: 254414 seconds Block delay: 32
Foo.csdf(5) from: 0x0000000000000000000000000000000000020000 Time delay: 303345 seconds Block delay: 53562
Foo.bsdf(4369999) from: 0x0000000000000000000000000000000000030000 Time delay: 303345 seconds Block delay: 57123
Foo.bsdf(115792089237316195423570985008687907853269984665640564039457584007913129639932) from: 0x0000000000000000000000000000000000030000 Time delay: 361136 seconds Block delay: 53562
Foo.bsdf(4370001) from: 0x0000000000000000000000000000000000020000 Time delay: 401699 seconds Block delay: 30011
Foo.asdf(98404317591302697895728486117905658464611205800369243083584335400803065093621) from: 0x0000000000000000000000000000000000020000 Time delay: 405856 seconds Block delay: 55538
Foo.asdf(82181221751846409617278589711486435926481222565914779440220456813086146204352) from: 0x0000000000000000000000000000000000010000 Time delay: 77600 seconds Block delay: 35334
Foo.csdf(79872180235038370578240087930809364320283939460722175098901465838627851118130) from: 0x0000000000000000000000000000000000020000 Time delay: 361136 seconds Block delay: 53562
Foo.csdf(115792089237316195423570985008687907853269984665640564039457584007913129639931) from: 0x0000000000000000000000000000000000020000 Time delay: 338920 seconds Block delay: 7323
Foo.bsdf(41661192703382420058606248607496353973014862028674264586095731523835873884449) from: 0x0000000000000000000000000000000000020000 Time delay: 156190 seconds Block delay: 6234
Foo.csdf(115792089237316195423570985008687907853269984665640564039457584007913129639929) from: 0x0000000000000000000000000000000000030000 Time delay: 271957 seconds Block delay: 5
Foo.bsdf(7) from: 0x0000000000000000000000000000000000020000 Time delay: 66543 seconds Block delay: 255
Foo.asdf(56460771626007978680176703879577391580640111980800679099119100744772346902825) from: 0x0000000000000000000000000000000000020000 Time delay: 338920 seconds Block delay: 255
Foo.bsdf(4369999) from: 0x0000000000000000000000000000000000020000 Time delay: 4177 seconds Block delay: 5022
Foo.csdf(1524785993) from: 0x0000000000000000000000000000000000020000 Time delay: 297507 seconds Block delay: 36859
Foo.bsdf(91191941997648570331845782433300455060714231629880791567007886618864267748840) from: 0x0000000000000000000000000000000000010000 Time delay: 19029 seconds Block delay: 7323
Foo.asdf(1) from: 0x0000000000000000000000000000000000030000 Time delay: 4177 seconds Block delay: 2
Foo.asdf(15455063137949734296523602836120957796090026793672118834464526419900710503005) from: 0x0000000000000000000000000000000000030000 Time delay: 166184 seconds Block delay: 42229
Foo.csdf(115792089237316195423570985008687907853269984665640564039457584007913129639934) from: 0x0000000000000000000000000000000000020000 Time delay: 358061 seconds Block delay: 4896
Foo.bsdf(966) from: 0x0000000000000000000000000000000000020000 Time delay: 405856 seconds Block delay: 42985
Foo.bsdf(115792089237316195423570985008687907853269984665640564039457584007913129639932) from: 0x0000000000000000000000000000000000020000 Time delay: 295050 seconds Block delay: 53011
Foo.asdf(7) from: 0x0000000000000000000000000000000000020000 Time delay: 434894 seconds Block delay: 53349
Foo.bsdf(52841453647672496968826193256958640800734554231771408205956281745404396507200) from: 0x0000000000000000000000000000000000010000 Time delay: 127 seconds Block delay: 32
Foo.bsdf(42952923681104225761253541795653292684036272771431273924024132718777897343624) from: 0x0000000000000000000000000000000000020000 Time delay: 33271 seconds Block delay: 12155
Foo.csdf(4369999) from: 0x0000000000000000000000000000000000010000 Time delay: 487078 seconds Block delay: 57086

[2024-06-12 11:23:18.33] [Worker 0] New coverage: 256 instr, 1 contracts, 1 seqs in corpus
[2024-06-12 11:23:20.83] [Worker 0] Test limit reached. Stopping.
[2024-06-12 11:23:20.83] [status] tests: 1/1, fuzzing: 101/50000, values: [], cov: 256, corpus: 1
echidna_asdf: failed!💥  
  Call sequence:
    Foo.asdf(1)
    Foo.bsdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)

Traces: 



Unique instructions: 256
Unique codehashes: 1
Corpus size: 1
Seed: 3826264006055040584

@gustavo-grieco
Copy link
Copy Markdown
Collaborator

gustavo-grieco commented Jun 12, 2024

Can we signal the user how big the sequence call at each step of the shrinking?

@samalws-tob
Copy link
Copy Markdown
Collaborator Author

samalws-tob commented Jun 12, 2024

We could do that, how often do you think we should log? If we log every shrinking step we'd end up with thousands of log messages, maybe we could log every 100 steps or every 1000 steps, or every time the size decreases significantly

@samalws-tob samalws-tob marked this pull request as ready for review June 12, 2024 17:41
@gustavo-grieco
Copy link
Copy Markdown
Collaborator

gustavo-grieco commented Jun 12, 2024

The shrink log should output something if the size of the sequence of transaction is reduced. Normally, it should not be more than 100 steps (max).

@gustavo-grieco
Copy link
Copy Markdown
Collaborator

gustavo-grieco commented Jun 13, 2024

Let me merge this one and we can talk about other small fixes later.

@gustavo-grieco gustavo-grieco merged commit 9f05340 into master Jun 13, 2024
@gustavo-grieco gustavo-grieco deleted the showTxsInTextMode branch June 13, 2024 09:18
@BrewTestBot BrewTestBot mentioned this pull request Jul 16, 2024
Merged
datradito pushed a commit to datradito/echidna-mcp that referenced this pull request Dec 29, 2025
@samalws-tob
show transactions when test is falsified in text mode (crytic#1271)
a6928b4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gustavo-grieco gustavo-grieco Awaiting requested review from gustavo-grieco gustavo-grieco is a code owner

@arcz arcz Awaiting requested review from arcz arcz is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Consider displaying assertion failure immediately once falsified in text mode

2 participants

@samalws-tob @gustavo-grieco