Repository navigation
Conversation
…n channel run_checks treated a live test whose sample was deleted upstream (every backend NOT_FOUND) like a capability failure, so check/doctor reported the channel broken. Such an action is now marked stale: it stays unverified instead of failing the check, while genuine and mixed failures still report broken.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #889
agent-reach check bilibilireported read and subtitle as broken becausetheir live_test fixture (BV1GJ411x7h7) has been deleted upstream: every
backend answered NOT_FOUND,
run_checkstreated that like a capabilityfailure, and doctor flipped the channel to "broken".
A live test whose every backend answers NOT_FOUND only proves the sample
itself is gone; it says nothing about whether the capability works. This
change teaches
run_checksthat distinction, in the dispatcher ratherthan in one channel, because the same failure mode applies to every
channel whose live_test names a specific resource (a video, a topic, a
repo, an account — see youtube, v2ex, github, twitter, instagram,
facebook, xueqiu).
What changes:
run_checks, an action that failed on all backends with NOT_FOUNDas the only error code is marked stale: its result rows keep
passed=falseand gain"stale": true, the action does not fail thecheck, and it is persisted as unverified rather than passed or failed.
Doctor therefore no longer reports the channel broken, the stale
actions stay out of
verified_actions, and a deleted sample no longercounts as a regression when the action passed before.
check --textprints STALE for those rows and appends"; N skipped (live sample no longer exists)" to the summary. The exit
code follows the remaining real results.
or a mix of NOT_FOUND with a real error still fails the action and
marks the channel broken, and a NEED_LOGIN failure on a sibling action
still reports needs_login.
The bilibili fixture itself is deliberately left in place: with stale
handling, a dead sample no longer poisons the verdict for any channel,
and swapping in a different video would only restart the clock until
that one is deleted too. The STALE rows keep the dead fixture visible so
it can be refreshed deliberately. The dynamic-sample alternative from
#889 was not chosen: it needs a new per-channel resolution hook in the
check path, adds a hot-list call (with its own failure modes) to every
bilibili check, and a random trending video is not guaranteed to have
subtitles, which would trade one false verdict for another.
Tests: new
tests/test_check_stale_sample.pycovers the stale case —including end-to-end on the real BilibiliChannel with mocked transports
(bili-cli not_found envelope + API -404) — the regression interaction,
genuine and mixed failures, the NEED_LOGIN interaction, and the --text
rendering. On the unpatched code the five stale tests fail and the two
preservation tests pass; with this change all seven pass. Full suite:
530 passed.
ruff check agent_reach testsclean.Live check note, honestly: I ran
agent-reach add bilibiliandagent-reach check bilibili --texton this machine before and after thechange. This network cannot complete a healthy bilibili check —
bili-cli's requests fail TLS negotiation (SSL WRONG_VERSION_NUMBER) and
the view endpoint answers HTTP 412 — so both runs report the same
environment failures (UPSTREAM_BROKEN, still FAIL, exit 1; attached
below). That also confirms the fix does not mask real connectivity
failures. The API itself is reachable from here (popular returns
code 0, and the deleted fixture returns Bilibili's 404 page). The
NOT_FOUND path is covered by the mocked-transport test above; on an
unrestricted network the expected output for bilibili today is
STALE rows for read/subtitle and "2/4 actions working; 2 skipped
(live sample no longer exists)" with exit 0.