Skip to content

Hypothesis shrinking doesn't find minimal example #1857

@boustrophedon

Description

@boustrophedon

As I mentioned in IRC, I had a dumb idea: Use hypothesis stateful testing to solve the towers of Hanoi problem.

It actually works:

Falsifying example: run_state_machine(factory=HanoiSolverSM, data=data(...))
state = HanoiSolverSM()
state.move(data=data(...))
Draw 1: (0, 1)
state.move(data=data(...))
Draw 2: (0, 2)
state.move(data=data(...))
Draw 3: (1, 2)
state.move(data=data(...))
Draw 4: (0, 1)
state.move(data=data(...))
Draw 5: (2, 0)
state.move(data=data(...))
Draw 6: (0, 2)
state.move(data=data(...))
Draw 7: (2, 0)
state.move(data=data(...))
Draw 8: (2, 1)
state.move(data=data(...))
Draw 9: (0, 1)
state.teardown()

but 9 moves is not minimal (it takes 2^n-1 moves for n rings). You can see that in draws 6 and 7 it just shuffles one ring back and forth.

In other runs, it does find a minimal example.
Full repro here: https://gist.github.com/boustrophedon/a0905a18ef45dea4a5b36b6cead54222


Also, I was wondering if there's a better way to structure this (maybe with the composite decorator?) such that instead of printing the draws it could just print move(0,1) somehow. There's a hook to print something extra on the line when you data.draw, so maybe I could implement a printer parameter for draw that takes a function with the results of the draw as input and prints it out/returns a string?

Metadata

Metadata

Assignees

No one assigned

    Labels

    test-case-reductionabout efficiently finding smaller failing examples

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions