Skip to content

CI: network calls and retry loops outside #681 still have no per-attempt bound #735

Open

Description

@cdeust

Issue #681 bounded each attempt of the pgvector apt retry loop (PR #712). The independent review of #712 found the same defect class elsewhere: network calls and retry loops with no per-attempt bound, so a stalled mirror holds a CI job until the job timeout instead of failing the attempt and retrying.

Sites read at origin/main 02c3cf0 (review evidence, not yet re-verified one by one):

curl without --max-time

Python download retry loops without a per-attempt bound

  • .github/actions/test-suite/action.yml:201, :218, :258
  • .github/workflows/ci.yml:183, :206, :436, :587

apt without a bound

  • scripts/setup.sh:115-116, :175

Already bounded, for reference: .github/workflows/upstream-identity.yml:54-55, pg_isready at .github/actions/test-suite/action.yml:146.

Expected: every attempt has a bound, a final failure fails the step (no fallback, no continue-on-error), the final message names the cause of the last attempt (the defect fixed in #712 round 2), and a test runs each loop with stubs, without a clock.

Caveat from the review: an apt-get blocked in uninterruptible sleep stays unbounded with or without timeout -k; nothing here was run on a real Ubuntu runner.

Activity

  1. added
    bugSomething isn't working
    github_actionsPull requests that update GitHub Actions code
    on Oct 12, 2026
  2. cdeust commented on Oct 28, 2026

    @cdeust
    OwnerAuthor

    CI autopilot : endroits revérifiés sur origin/main 8e1c131, et diff proposé

    L'inventaire de l'issue date de 02c3cf0 ; .github/workflows/ci.yml n'existe plus. Sites encore sans borne par tentative :

    Site Constat
    .github/actions/test-suite/action.yml:84 curl -fsSL de la clé PGDG sans --max-time ni retry (le même step que #681 borne pourtant apt-get à timeout ${APT_TIMEOUT}s)
    .github/actions/test-suite/action.yml:205, :222, :262 boucles de 5 tentatives (HF, FlashRank, tree-sitter) : python -c … sans timeout, le message final ne dit pas pourquoi la dernière tentative a échoué
    .github/workflows/release.yml:508 curl -fsSL de mcp-publisher sans --max-time
    .github/workflows/pipeline-post-merge.yml:95 urlopen(request) sans timeout=
    scripts/refresh_rustup_hash.sh:9 curl -sSf sans --max-time (script de maintenance manuel, pas exécuté en CI)
    scripts/setup.sh:115-116 apt-get sans borne (installeur utilisateur, pas exécuté en CI)

    Tout ce qui touche la CI est sous .github/, et le token de cette routine n'a pas la permission workflows. Je n'y touche donc pas. Voici le diff proposé, dans l'esprit de #712 : une borne par tentative, un échec final qui fait échouer le step, et un message qui donne la cause de la dernière tentative.

    --- a/.github/actions/test-suite/action.yml
    +++ b/.github/actions/test-suite/action.yml
    @@ -84,2 +84,3 @@
    -        sudo curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc \
    +        sudo curl -fsSL --connect-timeout 10 --max-time 60 \
    +          --retry 3 --retry-all-errors https://www.postgresql.org/media/keys/ACCC4CF8.asc \
               -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc
    @@ -205,7 +206,12 @@
    +        last=""
             for attempt in 1 2 3 4 5; do
    -          python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2', device='cpu')" && exit 0
    -          echo "HF pre-download attempt ${attempt} failed; retrying in $((attempt * 10))s" >&2
    +          timeout -k 10 300 python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2', device='cpu')" && exit 0
    +          status=$?
    +          if [ "$status" -eq 124 ]; then last="timed out after 300s"; else last="exit ${status}"; fi
    +          echo "HF pre-download attempt ${attempt}/5 ${last}; retrying in $((attempt * 10))s" >&2
               sleep $((attempt * 10))
             done
    -        echo "HF pre-download failed after 5 attempts" >&2
    +        echo "HF pre-download failed after 5 attempts (last attempt: ${last})" >&2
             exit 1
    # (même transformation pour FlashRank :222 et tree-sitter :262)
    --- a/.github/workflows/release.yml
    +++ b/.github/workflows/release.yml
    @@ -508 +508 @@
    -          curl -fsSL -o mcp-publisher.tar.gz \
    +          curl -fsSL --connect-timeout 10 --max-time 120 --retry 3 --retry-all-errors -o mcp-publisher.tar.gz \
    --- a/.github/workflows/pipeline-post-merge.yml
    +++ b/.github/workflows/pipeline-post-merge.yml
    @@ -95 +95 @@
    -                      with urlopen(request) as response:
    +                      with urlopen(request, timeout=30) as response:

    Les valeurs 300 s, 120 s et 30 s sont des propositions, pas des mesures : il faut les caler sur la durée observée des steps dans les runs Pipeline récents. Pour le test « sans horloge » demandé, le plus propre est d'extraire la boucle dans un scripts/ci_retry.sh appelé par l'action, testable avec un stub de timeout et de sleep. Cela demande quand même de modifier l'action.

    Décision humaine requise : appliquer ce diff (ou donner à la routine la permission workflows), et dire si les deux scripts hors CI (refresh_rustup_hash.sh, setup.sh) doivent rester dans le périmètre de cette issue. Je pose autopilot:hold. Pas vérifié : rien n'a tourné sur un runner Ubuntu.


    Generated by Claude Code

  3. added
    autopilot:holdCI autopilot stood down: a human decision is required
    on Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    autopilot:holdCI autopilot stood down: a human decision is requiredbugSomething isn't workinggithub_actionsPull requests that update GitHub Actions code

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions