To explain what is going on, I am going to use a silly example, just to keep it short. Suppose random.html is a pointless page which displays 1 or 2 at random, and we test it like this.

  Scenario Outline: Unreliable test
    When I load "random.html"
    Then I should see "<expected>"
    Examples:
      | expected |
      | 1        |
      | 2        |

Suppose that we run this, and get the results:
Unreliable test #1 -> PASS
Unreliable test #2 -> FAIL

Suppose we are using the --rerun option to the test-runner. When we do the re-run, it correctly only runs the second example. However, in the result, it names it #1. That is

Unreliable test #1 -> PASS (this did correctly check for and find 2 in the page).

This makes it difficult to link the result of the re-run to the original failure, which is leading to occasional false-positives from our CI system, which is a minor annoyance.

I am not sure that this is actually a bug, but I thought I woudl raise it.

I don't know much about Behat internals. Issue #154 might be related? If this is not considered to be a Behat probelm we can probably change our CI scripts which handle the test results to deal with our issue. Thank you for considering this.