Thanks @Thanatos9404, this is solid work. I reviewed it and found no bugs in the code: every evaluator gate I checked now uses the tolerant test, and the new tests pass.

#417 is in the 1.1.0 milestone, and we're holding 1.0.1 to what's already in flight, so I've moved this PR to 1.1.0. We'll pick it up right after 1.0.1 ships. If you want to get ahead of it, these are the items I'd want before merging:

1. Docs that still state the bare >= rule. EvalResult.matched is documented as overall_score >= match_threshold in API-Reference/auto.md:52, Guides/Evaluation/understanding-results.md:49 and threshold-gated-evaluation.md:206. The field verdict is similarity >= threshold in Getting-Started/thresholds-and-metrics.md:41, hungarian-matching.md:33 and :198, classification-logic.md:42 and :104, and models/confidence/README.md:28. A sentence on the tolerance in one place, linked from the others, would do.
2. The "Good Match" Laptop example in threshold-gated-evaluation.md:101. It still shows name as a TP at about 0.9, but your test asserts it's an FD (Levenshtein gives 0.4) with a products[0].name non-match.
3. "At every gate" in classification-logic.md:197 and the CHANGELOG. BaseComparator.binary_compare and ComparisonInfo.compare / ComparableFieldConfig.compare still use bare >=. Either switch them too or narrow the claim.
4. src/stickler/algorithms/README.md:90 still describes a difference between calculate_metrics and unordered_list_metrics that this PR removes.

Optional: structured_list_comparator.py:137 and hungarian.py:330 could call ThresholdHelper.apply_threshold_logic instead of re-counting. If the predicate lived in a dependency-free module, hungarian.py wouldn't need the deferred import either. Neither one blocks the merge.

Thanks for the careful review. bd338aa adds a short "Meeting a Threshold" section to thresholds-and-metrics.md and links it from each page you listed (plus Guides/Evaluation/README.md and the architecture page), fixes the Laptop example so name is an FD at 0.4 inside a TP pair with products[0].name in non_matches, and updates the algorithms README. binary_compare, ComparisonInfo.compare and ComparableFieldConfig.compare now use the same tolerant test, with a new test at 0.7 + 0.1 against 0.8, and the CHANGELOG names the two thresholds that stay exact (the ANLS* leaf cutoff and the bounding box IoU thresholds). I left the two optional refactors out to keep this one focused, and I noticed the Result Structure counts further down that page were already off before this PR (with 3 GT and 3 predictions, Hungarian pairs Cable with New Product as an FD), so I can do either of those in a separate PR if you like.

Thanks @Thanatos9404, nice turnaround. I re-checked bd338aa. The full suite passes, and so does test_threshold_tolerance.py (54 passed). Three of the four items are done, and the fourth is mostly done:

1. Bare >= wording: done. All eight places link to the new "Meeting a Threshold" section.
2. Laptop example: the pair itself is now right (a TP, with name an FD at 0.4). The rest of that worked example still needs a pass, see below.
3. "At every gate": done. binary_compare, ComparisonInfo.compare and ComparableFieldConfig.compare use the tolerant test, and the claim is scoped to the listed gates.
4. algorithms/README.md: done.

Before we merge this in 1.1.0:

• threshold-gated-evaluation.md:126 and the "Unmatched Items" scenario: with 3 ground-truth and 3 predicted products, every item is paired, so there is no FN or FA. Running the page's own Order/Product example on this branch gives products tp=1, fd=2, fn=0, fa=0, Mouse at 0.6725 (not 0.3), and Cable vs New Product as a paired FD at 0.133. The page shows 1/1/1/1 and cm_precision 0.5 (actual 1/3). Regenerating the block from a real run would fix it, and a test on the whole example would keep it fixed.
• Docstrings that say "the one every evaluator gate uses" (hungarian.py:293, base.py:244, algorithms/README.md): the ANLS* leaf threshold and bbox IoU stay exact, as your CHANGELOG says. "Every classification gate listed in Meeting a Threshold" would match.
• The heading "Good Match (similarity >= 0.8)" (threshold-gated-evaluation.md:93) sits over a 0.7999999999999999 example. "Good Match (meets 0.8)" avoids the contradiction.

Optional: facade.py:144 still uses a bare raw_sim < threshold for the explain verdict, which can't misfire today because of the score == 0.0 guard. ThresholdHelper.is_below_threshold would keep it consistent. The predicate move and the apply_threshold_logic reuse are still welcome but not needed.

One thing on our side, not yours: this is marked Breaking and milestoned 1.1.0, and RELEASING.md says breaking changes go in a major. We'll sort out the release labelling before it ships.

Thanks for checking it again. 0d971ca puts the example inputs on the page and replaces the Result Structure block with the counts and non_matches from a real run (tp=1, fd=2, Mouse at 0.6725, Cable and New Product as a paired FD at 0.133), and the Unmatched Items scenario now explains that an item is only left unmatched when one list is longer. The test now runs the whole example, plus the FN and FA cases you get by dropping one item. The docstrings now point at the gates listed in Meeting a Threshold, the heading reads "Good Match (meets 0.8)", and explain() uses is_below_threshold. I also merged dev (d19dc9b): the new list reporting verdicts go through the same helper, and two tests from #364 that pinned the exact scalar threshold now expect the tolerance.

Thanks for merging dev in. I checked the result: facade.py still uses the tolerant test in both places, and the structured_model.py hunk dropping out is expected, since #398 removed _calculate_object_level_metrics. None of the newer dev changes add a bare >= threshold check.

Security Scan Results

ASH Security Scan Report

• Report generated: 2026-10-28T22:50:24+00:00
• Time since scan: 3 minutes

Scan Metadata

• Project: ASH
• Scan executed: 2026-10-28T22:46:40+00:00
• ASH version: 3.1.2

Summary

Scanner Results

The table below shows findings by scanner, with status based on severity thresholds and dependencies:

• Severity levels:
  • Suppressed (S): Findings that have been explicitly suppressed and don't affect scanner status
  • Critical (C): Highest severity findings that require immediate attention
  • High (H): Serious findings that should be addressed soon
  • Medium (M): Moderate risk findings
  • Low (L): Lower risk findings
  • Info (I): Informational findings with minimal risk
• Duration (Time): Time taken by the scanner to complete its execution
• Actionable: Number of findings at or above the threshold severity level that require attention
• Result:
  • PASSED = No findings at or above threshold
  • FAILED = Findings at or above threshold
  • MISSING = Required dependencies not available
  • SKIPPED = Scanner explicitly disabled
  • ERROR = Scanner execution error
• Threshold: The minimum severity level that will cause a scanner to fail
  • Thresholds: ALL, LOW, MEDIUM, HIGH, CRITICAL
  • Source: Values in parentheses indicate where the threshold is set:
    • global (global_settings section in the ASH_CONFIG used)
    • config (scanner config section in the ASH_CONFIG used)
    • scanner (default configuration in the plugin, if explicitly set)
• Statistics calculation:
  • All statistics are calculated from the final aggregated SARIF report
  • Suppressed findings are counted separately and do not contribute to actionable findings
  • Scanner status is determined by comparing actionable findings to the threshold

Top 3 Hotspots

Files with the highest number of security findings:

Detailed Findings

Secret of type Hex High Entropy String detected

uuid.UUID(int=1),
            pathlib.Path("/tmp/x"),
            b"bytes",

def published_label(node: dict, section: str, object_lists: set) -> str:
            counts_objects = eval(  # noqa: S307
                cond.group(1),
                {},
                {"data": node, "section": section, "OBJECT_LISTS": object_lists},
            )
            return eval(unit.group(1), {}, {"counts_objects": counts_objects})  # noqa: S307

)
            return eval(unit.group(1), {}, {"counts_objects": counts_objects})  # noqa: S307

with contextlib.redirect_stdout(out):
        exec(code, namespace)
    prints = [line for line in code.splitlines() if line.startswith("print(")]

Report generated by Automated Security Helper (ASH) at 2026-10-28T22:50:26+00:00

Updated the description: binary_compare, both compare methods and explain() are now listed under the changes, and the left-alone list is down to the ANLS* leaf threshold and bbox IoU. Thanks for the careful reviews, and for filing #462.