Skip to content

[dynamo] Support bytearray.append() and extend() - #199675

Closed
jayzuccarelli wants to merge 6 commits into
pytorch:mainfrom
jayzuccarelli:dynamo-bytearray-append-extend
Closed

jayzuccarelli wants to merge 6 commits into
pytorch:mainfrom
jayzuccarelli:dynamo-bytearray-append-extend

Conversation

@jayzuccarelli

@jayzuccarelli jayzuccarelli commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

ByteArrayVariable had no append/extend, so calling either on a bytearray
graph broke. This adds both to tp_methods.

Both follow the existing in-place ops on ByteArrayVariable (+=, *=, %=):
build the result in a new buffer, record the mutation with side_effects,
and swap self.data. For a bytearray that came in as an input, the existing
bytearray_mutation replay handler writes the result back with old[:] = new
after the graph, so tracing never mutates the live object.

append runs the item through PyNumber_Index (so index objects work)
and lets the real bytearray.append raise the range error. extend takes
constants (bytes, bytearray, str, lists of ints, ...) directly and lets
the real bytearray.extend produce CPython's errors, including the str
special case. Other iterables (generators, map, lists holding index
objects) are unpacked item by item through PyNumber_Index; a non-iterable
raises "can't extend bytearray with X". If an item is not a constant after
PyNumber_Index the call is declined as before.

Removes 4 expected-failure markers that now pass:
CPython313-test_bytes-ByteArrayTest.test_append, test_extend,
test_exhausted_iterator, and CPython313-test_builtin-BuiltinTest.test_bytearray_extend_error.

Part of #192126

Test Plan:

python test/dynamo/test_bytearray.py
PYTORCH_TEST_WITH_DYNAMO=1 python test/cpython/v3_13/test_bytes.py
PYTORCH_TEST_WITH_DYNAMO=1 python test/cpython/v3_13/test_builtin.py

New tests: test_append, test_extend and test_append_extend_arg_mutation fail
on main and pass with this change; test_append_graph_break covers write-back
across a graph break. I also ran every test/cpython/v3_13 file under
PYTORCH_TEST_WITH_DYNAMO=1 before and after; the only new unexpected
successes were the 4 markers removed here.

Fix written with AI assistance (Claude), reviewed and tested by me.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @kadeng @chauhang @amjames @jataylo @azahed98

@pytorch-bot

pytorch-bot Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

PR Status: Approved 🚀

Please fix all CI failures and trigger merge by commenting "@pytorchbot merge".

✅ AUTOMATED REVIEW - OUTDATED (earlier commit) - Ready for human review:

Adds Dynamo tracing for bytearray.append/extend and removes four now-passing
CPython expected-failure markers. All three of williamwen42's review comments
are addressed at HEAD: bytes subclasses are declined, items are read one at a
time, and the length hint is called. Each fix has a fullgraph regression test.
One narrow buffer-precedence gap remains for bytearray subclasses.

Findings (1):

[minor] torch/_dynamo/variables/lists.py:2442
Buffer-precedence decline only covers bytes subclasses. A bytearray subclass
becomes a UserDefinedObjectVariable, so if it defines Python __iter__ and
__len__, extend traces __iter__: dst.extend(BA(b'ab')) gives b'qt' compiled vs
b'qab' eager, with no graph break. Decline (bytes, bytearray) subclasses too, or
any non-ByteArrayVariable buffer exporter.

Reviewed commit: 37f6994 (NOT the current head cabbb56)

Review run

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/199675

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit cabbb56 with merge base 0877d49 (image):

NEW FAILURES SUPPRESSED BY AI (non-blocking) - The following job failed, but the AI CI Advisor judged the failure not to be evidence against this change:

  • pull / linux-jammy-aarch64-py3.11-clang21 / build (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
    /__w/pytorch/pytorch/aten/src/ATen/native/quantized/cpu/qembeddingbag.cpp:222:20: error: function 'embedding_neon_kernel' is not needed and will not be emitted [-Werror,-Wunneeded-internal-declaration]

    AI verdict: AI verdict: not related

    The suspect commit only changes Dynamo Python files (torch/_dynamo/variables/lists.py and a test). The build fails with a clang -Wunneeded-internal-declaration error in qembeddingbag.cpp. That error comes from the PR's base being older than ada3fee (AArch64: Enable clang build for PR's #198642), the trunk commit that added this aarch64 clang21 job and fixed the warning with [[maybe_unused]].

    Full reasoning on HUD →

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot Bot added ciflow/inductor ciflow/torchtitan Run TorchTitan integration tests module: dynamo labels Oct 4, 2026
@pytorch-bot

pytorch-bot Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Workflows were awaiting approval. CI has now been triggered for the ciflow labels on this PR.

@pytorch-bot

pytorch-bot Bot commented Oct 4, 2026

Copy link
Copy Markdown

The following ciflow label(s) have been added but CI has not been triggered yet because the workflows are awaiting approval:

  • ciflow/inductor
  • ciflow/torchtitan

Once a maintainer approves the workflows (scroll to the bottom of the PR page), the corresponding CI jobs will be triggered automatically. Please ping one of the reviewers if you do not have access to approve and run workflows.

@jayzuccarelli
jayzuccarelli marked this pull request as ready for review October 4, 2026 11:45
@jayzuccarelli

Copy link
Copy Markdown
Contributor Author

@pytorchbot label "release notes: dynamo"

@guilhermeleobas

Copy link
Copy Markdown
Collaborator

@pytorchbot pre-review accept

@guilhermeleobas guilhermeleobas added triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module and removed bot-triage-error labels Oct 5, 2026
@guilhermeleobas

Copy link
Copy Markdown
Collaborator

@pytorchbot pre-review accept

@pytorch-bot pytorch-bot Bot added the in progress PR is being worked on by the author, until the automated review accepts it label Oct 5, 2026
@guilhermeleobas guilhermeleobas added in progress PR is being worked on by the author, until the automated review accepts it and removed in progress PR is being worked on by the author, until the automated review accepts it labels Oct 7, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review of e5248c5: changes requested before human review.

Adds Dynamo support for bytearray.append/extend and removes four CPython expected failures that now pass. The constant fast path in bytearray_extend gives silently wrong results for a fresh list/tuple iterator, because it reads a copy and never advances the tracked iterator. Two smaller issues remain: the range check runs only after the whole input has been consumed, and one new test passes even without the change.

🔴 1 blocking finding is attached to the code below.

⚪ 2 non-blocking findings

tx, iterable, lambda item: items.append(pynumber_index(tx, item))

The byte range (0..255) is checked only in new_data.extend(values), after the whole input has been drained. CPython stops at the first out-of-range item. So after a caught ValueError the generator is left in a different state, a later exception from the generator wins over the ValueError, and an unbounded input such as itertools.count(250) never finishes tracing. Check the range inside the per-item callback instead.


def test_append_graph_break(self):

test_append_graph_break uses torch.compile(backend="eager") without fullgraph or a CompileCounter, so it also passes without this PR: append just graph-breaks and runs eagerly. Assert frame/op counts so the test checks that the traced mutation is replayed.

Please address the findings and push; a new commit normally gets a fresh automated review.

This review is AI-generated and advisory. Each new automated review replaces the previous one.

Comment thread torch/_dynamo/variables/lists.py Outdated
if not self.is_mutable():
return None
iterable = args[0]
if iterable.is_python_constant():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 For a fresh ListIteratorVariable/TupleIteratorVariable (index 0), is_python_constant() is True and as_python_constant() returns a new iter() copy. extend reads the copy, so the tracked iterator is never advanced. Example: it = iter(\[1, 2\]); ba.extend(it); next(it, 'done') gives 1 under Dynamo but 'done' in eager, and an escaping iterator is reconstructed unconsumed. Exclude iterators from the fast path, or limit it to concrete containers as list_extend does (lists.py ~543), and add a regression test.

@jayzuccarelli

Copy link
Copy Markdown
Contributor Author

@guilhermeleobas addressed all three automated-review findings in 4eb1df4 (iterator consumption, per-item range check, stronger graph-break test). Could you approve CI so the review re-runs?

@github-actions github-actions Bot added ready for review Automated review passed, ready for a final review by a maintainer. and removed in progress PR is being worked on by the author, until the automated review accepts it labels Oct 8, 2026
github-actions[bot]

This comment was marked as outdated.

@github-actions
github-actions Bot dismissed their stale review October 8, 2026 16:21

Withdrawn: the automated review of ece0eaa found nothing blocking.

@jayzuccarelli

Copy link
Copy Markdown
Contributor Author

@guilhermeleobas pushed ece0eaa to fix the lint failure (a stray blank line in the lists.py imports). The rest is unchanged since 4eb1df4, which addresses your review. Can you approve the CI run when you have a moment?

@jayzuccarelli

Copy link
Copy Markdown
Contributor Author

@jansel @williamwen42 since @guilhermeleobas is out for a couple of weeks: this one already has his review addressed (4eb1df4) and just needs a CI approval after the lint fix (ece0eaa). Could one of you approve the workflow run and take a look? #199810 (insert/pop) builds on it.

@jansel
jansel requested a review from williamwen42 October 10, 2026 17:48

@williamwen42 williamwen42 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I manually reviewed AI-reported concerns, please address the P2 findings; P3 can be done here or in a followup.

# traced iterator unconsumed.
if isinstance(iterable, ConstantVariable):
values = iterable.as_python_constant()
elif vt_is_iterable(iterable):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. [P2] Preserve buffer precedence for bytes subclasses

import torch

class Raw(bytes):
    def __iter__(self):
        return iter((9,))

def f(x, value):
    y = x + 1
    dst = bytearray(b"q")
    dst.extend(value)
    return y, dst

x, value = torch.ones(1), Raw(b"ab")
print(f(x, value)[1])                                 # bytearray(b'qab')
print(torch.compile(f, backend="eager")(x, value)[1])  # bytearray(b'q\t')

CPython reads the buffer before considering __iter__. The new iterable branch invokes the subclass override instead, changing the result.

Suggested fix: Decline bytes subclasses before iterable dispatch until raw-buffer access is modeled. This restores eager fallback; strict fullgraph remains unsupported for these subclasses.

         if isinstance(iterable, ConstantVariable):
             values = iterable.as_python_constant()
+        elif issubclass(maybe_get_python_type(iterable), bytes):
+            return None
         elif vt_is_iterable(iterable):

Comment thread torch/_dynamo/variables/lists.py Outdated
)
values.append(value)

unpack_and_apply_fn(tx, iterable, add)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2. [P2] Validate each byte before consuming the next item

import torch

def f(x):
    y = x + 1
    it = iter([1, 300, 2])
    dst = bytearray()
    try:
        dst.extend(it)
    except ValueError:
        pass
    return y, next(it, "done")

x = torch.ones(1)
print(f(x)[1])                                                  # 2
print(torch.compile(f, backend="eager", fullgraph=True)(x)[1])    # done

The list/tuple iterator fast path in unpack_and_apply_fn drains the iterator before add validates bytes, losing the tail after an invalid byte. It also snapshots direct lists, missing growth during __index__.

Suggested fix: Advance and validate one item at a time, and decline immediately on nonconstant values. Import generic_getiter, pyiter_next, ObservedUserStopIteration, and handle_observed_exception for this loop.

-            unpack_and_apply_fn(tx, iterable, add)
-            if non_constant:
-                return None
+            iterator = generic_getiter(tx, iterable)
+            while True:
+                try:
+                    item = pyiter_next(tx, iterator)
+                except ObservedUserStopIteration:
+                    handle_observed_exception(tx)
+                    break
+                add(item)
+                if non_constant:
+                    return None

# traced iterator unconsumed.
if isinstance(iterable, ConstantVariable):
values = iterable.as_python_constant()
elif vt_is_iterable(iterable):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3. [P3] Preserve length-hint effects and errors

import torch

class Items:
    def __iter__(self):
        return iter((1,))
    def __length_hint__(self):
        raise ValueError("hint failed")

def f(x, items):
    y = x + 1
    dst = bytearray()
    try:
        dst.extend(items)
    except ValueError:
        return y, "caught"
    return y, dst

x = torch.ones(1)
print(f(x, Items())[1])                                                # caught
print(torch.compile(f, backend="eager", fullgraph=True)(x, Items())[1])  # bytearray(b'\x01')

CPython calls the original argument's __len__ or __length_hint__ after obtaining the iterator and before consuming it. Dynamo skips this call, losing side effects and exceptions.

Suggested fix: Reuse the existing length-hint implementation on the original iterable with default 32. Add this before the loop in finding 2. This P3 can be handled here or in a follow-up.

             iterator = generic_getiter(tx, iterable)
+            variables.BuiltinVariable(operator.length_hint).call_function(
+                tx, [iterable, ConstantVariable.create(32)], {}
+            )
             while True:

@jayzuccarelli

Copy link
Copy Markdown
Contributor Author

Thanks @williamwen42! The 30 red jobs all failed at checkout (/__w/pytorch/pytorch/.git: Permission denied) before building, so they look like a runner issue. Could you re-run the failed jobs? I'll send @pytorchbot merge once they're green.

@williamwen42

Copy link
Copy Markdown
Member

@pytorchbot rebase

@pytorchmergebot

Copy link
Copy Markdown
Collaborator

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

ByteArrayVariable had no append/extend, so calling either on a bytearray
graph broke. This adds both to tp_methods.

Both follow the existing in-place ops on ByteArrayVariable (+=, *=, %=):
build the result in a new buffer, record the mutation with side_effects,
and swap self.data. For a bytearray that came in as an input, the existing
bytearray_mutation replay handler writes the result back with old[:] = new
after the graph, so tracing never mutates the live object.

append runs the item through PyNumber_Index (so __index__ objects work)
and lets the real bytearray.append raise the range error. extend takes
constants (bytes, bytearray, str, lists of ints, ...) directly and lets
the real bytearray.extend produce CPython's errors, including the str
special case. Other iterables (generators, map, lists holding __index__
objects) are unpacked item by item through PyNumber_Index; a non-iterable
raises "can't extend bytearray with X". If an item is not a constant after
PyNumber_Index the call is declined as before.

Removes 4 expected-failure markers that now pass:
CPython313-test_bytes-ByteArrayTest.test_append, test_extend,
test_exhausted_iterator, and CPython313-test_builtin-BuiltinTest.test_bytearray_extend_error.

Part of pytorch#192126

Test Plan:

```
python test/dynamo/test_bytearray.py
PYTORCH_TEST_WITH_DYNAMO=1 python test/cpython/v3_13/test_bytes.py
PYTORCH_TEST_WITH_DYNAMO=1 python test/cpython/v3_13/test_builtin.py
```

New tests: test_append, test_extend and test_append_extend_arg_mutation fail
on main and pass with this change; test_append_graph_break covers write-back
across a graph break. I also ran every test/cpython/v3_13 file under
PYTORCH_TEST_WITH_DYNAMO=1 before and after; the only new unexpected
successes were the 4 markers removed here.

Fix written with AI assistance (Claude), reviewed and tested by me.
Python 3.12 raises a generic TypeError for bytearray.extend(str); the
"expected iterable of integers" wording is 3.13+. Check only the type.

Fix written with AI assistance (Claude), reviewed and tested by me.
… first bad byte

Only literal constants take the extend fast path now. Iterator variables are
also python constants, but reading one returns a copy, so the traced iterator
was never advanced (next(it) after extend still returned the first item).

The 0..255 range check now runs per item while the iterable is consumed, so a
bad byte stops iteration where CPython does instead of after draining it.

test_append_graph_break now counts compiled frames, so it fails if append or
extend graph-break instead of being traced.
Addresses the review on pytorch#199675.

bytearray.extend handed the argument to unpack_and_apply_fn, which differs
from CPython's bytearray_extend in three ways:

- A bytes subclass that overrides __iter__ was iterated. CPython checks for
  a buffer first and never calls __iter__, so the result differed. Raw-buffer
  access is not modeled here, so bytes subclasses now decline before the
  iterable branch.
- unpack_and_apply_fn drains list and tuple iterators before the per-item
  callback runs, so after a ValueError for an out-of-range byte the rest of
  the iterator was gone. It also snapshots a list, missing items appended by
  __index__. The loop now uses generic_getiter and pyiter_next and validates
  each item before reading the next one.
- CPython asks the argument for a length hint (default 32) after getting the
  iterator and before reading any item. That call was skipped, losing its
  side effects and exceptions. It is now made through operator.length_hint.

test_append_graph_break now has a tensor op on both sides of each mutation
and checks op_count, so it fails if append or extend graph-break instead of
being traced. Before, frame_count == 2 also held when extend fell back in
the resumed frame.

Test Plan:

```
python test/dynamo/test_bytearray.py
PYTORCH_TEST_WITH_DYNAMO=1 python test/cpython/v3_13/test_bytes.py ByteArrayTest
```

test_bytearray.py: 61 passed on Python 3.13. The four new extend tests fail
without this change. The CPython ByteArrayTest results are the same before
and after this change.

Written with AI assistance (Claude).
CPython's bytearray_extend takes the buffer path for any buffer exporter, so
a bytearray subclass that overrides __iter__ is extended from its buffer,
like a bytes subclass. The decline added in the previous commit only covered
bytes: a bytearray subclass with Python __iter__ and __len__ becomes a
UserDefinedObjectVariable and was iterated, giving b'qt' instead of b'qab'.

Decline bytes and bytearray subclasses alike. A plain bytearray argument is
a ByteArrayVariable and still takes the traced path.

Test Plan:

```
python test/dynamo/test_bytearray.py
```

61 passed on Python 3.13. test_extend_bytes_subclass_reads_the_buffer now
also runs a bytearray subclass, which fails without this change.

Written with AI assistance (Claude).
@pytorchmergebot

Copy link
Copy Markdown
Collaborator

Successfully rebased dynamo-bytearray-append-extend onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout dynamo-bytearray-append-extend && git pull --rebase)

@pytorchmergebot
pytorchmergebot force-pushed the dynamo-bytearray-append-extend branch from c204f1d to cabbb56 Compare October 15, 2026 22:35
@williamwen42

Copy link
Copy Markdown
Member

@pytorchbot merge

@pytorch-bot pytorch-bot Bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 16, 2026
@pytorchmergebot

Copy link
Copy Markdown
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot

Copy link
Copy Markdown
Collaborator

Merge failed

Reason: 1 jobs have failed, first few of them are: pull / linux-jammy-aarch64-py3.11-clang21 / build

Details for Dev Infra team

Raised by workflow job

@williamwen42

Copy link
Copy Markdown
Member

@pytorchbot merge -i

@pytorchmergebot

Copy link
Copy Markdown
Collaborator

Merge started

Your change will be merged while ignoring the following 1 checks: pull / linux-jammy-aarch64-py3.11-clang21 / build

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

izaitsevfb added a commit that referenced this pull request Oct 28, 2026
Dr.CI answers HTTP 400 for any exception it hits while recomputing. The
three retries ran back to back in the same second, so a short upstream
blip failed all of them; trymerge then fell back to the Dr.CI check run
summary, which deliberately drops AI_NOT_RELATED, and blocked merges on
failures the AI CI Advisor had cleared (#199675, #201134).

Retry after 5, 15 and 30 seconds instead, and print the first 500 bytes of
the response body, where Dr.CI names the exception.
pytorchmergebot pushed a commit that referenced this pull request Nov 3, 2026
✴️ iz2: Dr.CI's `/api/drci/drci` answers HTTP 400 for any exception it hits while recomputing. trymerge retried three times back to back, so a short upstream blip failed every attempt. It then fell back to the Dr.CI check run summary, which deliberately drops `AI_NOT_RELATED`, and blocked the merge on failures the AI CI Advisor had cleared. This happened on #199675 and #201134; both were later merged with `-i`.

This change, in `get_drci_classifications`:
- makes up to 4 attempts. After HTTP 400, 408, 429 or 5xx, a temporary DNS failure, a kernel connect timeout, or a connection dropped or cut off mid-response, it pauses 5, 15 and 30 seconds first. Refused connections, unknown hosts, no route (as on internal runners) and unexpected exceptions retry immediately;
- does not retry a call that sat silent for the whole socket timeout, an error body that stalls, or a 504 (HUD ran out its 900 s `maxDuration`);
- gives Dr.CI calls a 960 s socket timeout, so a stalled response can no longer hang the merge. `gh_fetch_url` gains an optional `timeout`; other callers are unchanged;
- logs `Attempt N of TOTAL` with the first 500 bytes of the error body, where Dr.CI names the exception, next to the existing `Error fetching ... HTTP Error 400` line.

Each call keeps its own retry state; during a full Dr.CI outage every failed call pays its ~50 s of pauses, and the merge loop's next iteration (`MERGE_RETRY_INTERVAL_SECONDS`, 5 min) tries again. Returning `None` on failure, the `From Dr.CI API: null` diagnostic and the stale-summary guard are unchanged.

In the last two days of merge jobs (72), 8 Dr.CI calls in 6 jobs hit a 400. Back-to-back retries recovered 7 of them.

Test plan: `python -m unittest test_trymerge` in `.github/scripts`, including new `TestDrciRetries` and `TestFetchUrlTimeout`.
Pull Request resolved: #202124
Approved by: https://github.com/huydhn
dominikjastrzebskix pushed a commit to dominikjastrzebskix/pytorch that referenced this pull request Nov 5, 2026
…#202124)

✴️ iz2: Dr.CI's `/api/drci/drci` answers HTTP 400 for any exception it hits while recomputing. trymerge retried three times back to back, so a short upstream blip failed every attempt. It then fell back to the Dr.CI check run summary, which deliberately drops `AI_NOT_RELATED`, and blocked the merge on failures the AI CI Advisor had cleared. This happened on pytorch#199675 and pytorch#201134; both were later merged with `-i`.

This change, in `get_drci_classifications`:
- makes up to 4 attempts. After HTTP 400, 408, 429 or 5xx, a temporary DNS failure, a kernel connect timeout, or a connection dropped or cut off mid-response, it pauses 5, 15 and 30 seconds first. Refused connections, unknown hosts, no route (as on internal runners) and unexpected exceptions retry immediately;
- does not retry a call that sat silent for the whole socket timeout, an error body that stalls, or a 504 (HUD ran out its 900 s `maxDuration`);
- gives Dr.CI calls a 960 s socket timeout, so a stalled response can no longer hang the merge. `gh_fetch_url` gains an optional `timeout`; other callers are unchanged;
- logs `Attempt N of TOTAL` with the first 500 bytes of the error body, where Dr.CI names the exception, next to the existing `Error fetching ... HTTP Error 400` line.

Each call keeps its own retry state; during a full Dr.CI outage every failed call pays its ~50 s of pauses, and the merge loop's next iteration (`MERGE_RETRY_INTERVAL_SECONDS`, 5 min) tries again. Returning `None` on failure, the `From Dr.CI API: null` diagnostic and the stale-summary guard are unchanged.

In the last two days of merge jobs (72), 8 Dr.CI calls in 6 jobs hit a 400. Back-to-back retries recovered 7 of them.

Test plan: `python -m unittest test_trymerge` in `.github/scripts`, including new `TestDrciRetries` and `TestFetchUrlTimeout`.
Pull Request resolved: pytorch#202124
Approved by: https://github.com/huydhn
dominikjastrzebskix pushed a commit to dominikjastrzebskix/pytorch that referenced this pull request Nov 9, 2026
…#202124)

✴️ iz2: Dr.CI's `/api/drci/drci` answers HTTP 400 for any exception it hits while recomputing. trymerge retried three times back to back, so a short upstream blip failed every attempt. It then fell back to the Dr.CI check run summary, which deliberately drops `AI_NOT_RELATED`, and blocked the merge on failures the AI CI Advisor had cleared. This happened on pytorch#199675 and pytorch#201134; both were later merged with `-i`.

This change, in `get_drci_classifications`:
- makes up to 4 attempts. After HTTP 400, 408, 429 or 5xx, a temporary DNS failure, a kernel connect timeout, or a connection dropped or cut off mid-response, it pauses 5, 15 and 30 seconds first. Refused connections, unknown hosts, no route (as on internal runners) and unexpected exceptions retry immediately;
- does not retry a call that sat silent for the whole socket timeout, an error body that stalls, or a 504 (HUD ran out its 900 s `maxDuration`);
- gives Dr.CI calls a 960 s socket timeout, so a stalled response can no longer hang the merge. `gh_fetch_url` gains an optional `timeout`; other callers are unchanged;
- logs `Attempt N of TOTAL` with the first 500 bytes of the error body, where Dr.CI names the exception, next to the existing `Error fetching ... HTTP Error 400` line.

Each call keeps its own retry state; during a full Dr.CI outage every failed call pays its ~50 s of pauses, and the merge loop's next iteration (`MERGE_RETRY_INTERVAL_SECONDS`, 5 min) tries again. Returning `None` on failure, the `From Dr.CI API: null` diagnostic and the stale-summary guard are unchanged.

In the last two days of merge jobs (72), 8 Dr.CI calls in 6 jobs hit a 400. Back-to-back retries recovered 7 of them.

Test plan: `python -m unittest test_trymerge` in `.github/scripts`, including new `TestDrciRetries` and `TestFetchUrlTimeout`.
Pull Request resolved: pytorch#202124
Approved by: https://github.com/huydhn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/inductor ciflow/torchtitan Run TorchTitan integration tests ciflow/trunk Trigger trunk jobs on your pull request Merged module: dynamo open source ready for review Automated review passed, ready for a final review by a maintainer. release notes: dynamo triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants