Skip to content

ate.scheduler.assignment.duration counts worker contention as an error #2243

Closed

Description

@JeffLuoo

What happened?

When two resumes race for the last free slot on a worker, the losing attempt records an observation on ate.scheduler.assignment.duration with ate.scheduler.outcome="error" and an error.type key. The retry then records a second observation with the real outcome, so one resume produces two records instead of one.

This contradicts the note on the instrument in the metric registry. The note says that ateapi records only the last attempt when a retry happens.

Expected Behavior

One resume gives one observation on ate.scheduler.assignment.duration, with the outcome of the attempt that ended the loop. A losing attempt in a race between concurrent resumes records no observation.

Out of scope

A resume can run out of retries because every attempt found a full worker. In that case, the outcome is error with error.type="Aborted", not no_capacity. #2259 tracks this case.

Metadata

Metadata

Labels

Type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions