Skip to content

Testing: raster image comparison #3727

@swharden

Description

@swharden

The goal of this issue is to track developing a graphical testing system. Regression tests can't use image hashes because slight differences in systems (e.g., installed fonts or sub-pixel alignment changes) cause hashes to mismatch when changes are minor. An ideal graphical testing system would evaluate how "much" an image changes, and ideally highlight where.

I recall hearing an interview with someone describing how matplotlib runs graphics tests. They generate a bunch of test images (like the ScottPlot cookbook), save them, then after making code changes they generate a new set of images and perform comparisons on the old vs. new images by creating a black and white difference image. The difference image can evaluate how much changed graphically (how large the difference is), and point the developer to exactly where in the image changed.

Cookbook and test images should not be stored in this repository, but a new tool should be developed to create and store them locally (perhaps in a named and dated folder) and facilitate comparisons as described above.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions