Repository navigation
[dynamo] Support bytearray.append() and extend() - #199675
jayzuccarelli wants to merge 6 commits into
Conversation
PR Status: Approved 🚀Please fix all CI failures and trigger merge by commenting "@pytorchbot merge". ✅ AUTOMATED REVIEW - OUTDATED (earlier commit) - Ready for human review:
Findings (1): Reviewed commit: 🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/199675
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit cabbb56 with merge base 0877d49 ( NEW FAILURES SUPPRESSED BY AI (non-blocking) - The following job failed, but the AI CI Advisor judged the failure not to be evidence against this change:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
|
|
The following ciflow label(s) have been added but CI has not been triggered yet because the workflows are awaiting approval:
Once a maintainer approves the workflows (scroll to the bottom of the PR page), the corresponding CI jobs will be triggered automatically. Please ping one of the reviewers if you do not have access to approve and run workflows. |
|
@pytorchbot label "release notes: dynamo" |
|
@pytorchbot pre-review accept |
|
@pytorchbot pre-review accept |
There was a problem hiding this comment.
Automated review of e5248c5: changes requested before human review.
Adds Dynamo support for bytearray.append/extend and removes four CPython expected failures that now pass. The constant fast path in bytearray_extend gives silently wrong results for a fresh list/tuple iterator, because it reads a copy and never advances the tracked iterator. Two smaller issues remain: the range check runs only after the whole input has been consumed, and one new test passes even without the change.
🔴 1 blocking finding is attached to the code below.
⚪ 2 non-blocking findings
pytorch/torch/_dynamo/variables/lists.py
Line 2438 in e5248c5
The byte range (0..255) is checked only in new_data.extend(values), after the whole input has been drained. CPython stops at the first out-of-range item. So after a caught ValueError the generator is left in a different state, a later exception from the generator wins over the ValueError, and an unbounded input such as itertools.count(250) never finishes tracing. Check the range inside the per-item callback instead.
pytorch/test/dynamo/test_bytearray.py
Line 423 in e5248c5
test_append_graph_break uses torch.compile(backend="eager") without fullgraph or a CompileCounter, so it also passes without this PR: append just graph-breaks and runs eagerly. Assert frame/op counts so the test checks that the traced mutation is replayed.
Please address the findings and push; a new commit normally gets a fresh automated review.
This review is AI-generated and advisory. Each new automated review replaces the previous one.
| if not self.is_mutable(): | ||
| return None | ||
| iterable = args[0] | ||
| if iterable.is_python_constant(): |
There was a problem hiding this comment.
🔴 For a fresh ListIteratorVariable/TupleIteratorVariable (index 0), is_python_constant() is True and as_python_constant() returns a new iter() copy. extend reads the copy, so the tracked iterator is never advanced. Example: it = iter(\[1, 2\]); ba.extend(it); next(it, 'done') gives 1 under Dynamo but 'done' in eager, and an escaping iterator is reconstructed unconsumed. Exclude iterators from the fast path, or limit it to concrete containers as list_extend does (lists.py ~543), and add a regression test.
|
@guilhermeleobas addressed all three automated-review findings in 4eb1df4 (iterator consumption, per-item range check, stronger graph-break test). Could you approve CI so the review re-runs? |
Withdrawn: the automated review of ece0eaa found nothing blocking.
|
@guilhermeleobas pushed ece0eaa to fix the lint failure (a stray blank line in the lists.py imports). The rest is unchanged since 4eb1df4, which addresses your review. Can you approve the CI run when you have a moment? |
|
@jansel @williamwen42 since @guilhermeleobas is out for a couple of weeks: this one already has his review addressed (4eb1df4) and just needs a CI approval after the lint fix (ece0eaa). Could one of you approve the workflow run and take a look? #199810 (insert/pop) builds on it. |
williamwen42
left a comment
There was a problem hiding this comment.
I manually reviewed AI-reported concerns, please address the P2 findings; P3 can be done here or in a followup.
| # traced iterator unconsumed. | ||
| if isinstance(iterable, ConstantVariable): | ||
| values = iterable.as_python_constant() | ||
| elif vt_is_iterable(iterable): |
There was a problem hiding this comment.
1. [P2] Preserve buffer precedence for bytes subclasses
import torch class Raw(bytes): def __iter__(self): return iter((9,)) def f(x, value): y = x + 1 dst = bytearray(b"q") dst.extend(value) return y, dst x, value = torch.ones(1), Raw(b"ab") print(f(x, value)[1]) # bytearray(b'qab') print(torch.compile(f, backend="eager")(x, value)[1]) # bytearray(b'q\t')CPython reads the buffer before considering
__iter__. The new iterable branch invokes the subclass override instead, changing the result.Suggested fix: Decline bytes subclasses before iterable dispatch until raw-buffer access is modeled. This restores eager fallback; strict fullgraph remains unsupported for these subclasses.
if isinstance(iterable, ConstantVariable): values = iterable.as_python_constant() + elif issubclass(maybe_get_python_type(iterable), bytes): + return None elif vt_is_iterable(iterable):
| ) | ||
| values.append(value) | ||
|
|
||
| unpack_and_apply_fn(tx, iterable, add) |
There was a problem hiding this comment.
2. [P2] Validate each byte before consuming the next item
import torch def f(x): y = x + 1 it = iter([1, 300, 2]) dst = bytearray() try: dst.extend(it) except ValueError: pass return y, next(it, "done") x = torch.ones(1) print(f(x)[1]) # 2 print(torch.compile(f, backend="eager", fullgraph=True)(x)[1]) # doneThe list/tuple iterator fast path in
unpack_and_apply_fndrains the iterator beforeaddvalidates bytes, losing the tail after an invalid byte. It also snapshots direct lists, missing growth during__index__.Suggested fix: Advance and validate one item at a time, and decline immediately on nonconstant values. Import
generic_getiter,pyiter_next,ObservedUserStopIteration, andhandle_observed_exceptionfor this loop.- unpack_and_apply_fn(tx, iterable, add) - if non_constant: - return None + iterator = generic_getiter(tx, iterable) + while True: + try: + item = pyiter_next(tx, iterator) + except ObservedUserStopIteration: + handle_observed_exception(tx) + break + add(item) + if non_constant: + return None
| # traced iterator unconsumed. | ||
| if isinstance(iterable, ConstantVariable): | ||
| values = iterable.as_python_constant() | ||
| elif vt_is_iterable(iterable): |
There was a problem hiding this comment.
3. [P3] Preserve length-hint effects and errors
import torch class Items: def __iter__(self): return iter((1,)) def __length_hint__(self): raise ValueError("hint failed") def f(x, items): y = x + 1 dst = bytearray() try: dst.extend(items) except ValueError: return y, "caught" return y, dst x = torch.ones(1) print(f(x, Items())[1]) # caught print(torch.compile(f, backend="eager", fullgraph=True)(x, Items())[1]) # bytearray(b'\x01')CPython calls the original argument's
__len__or__length_hint__after obtaining the iterator and before consuming it. Dynamo skips this call, losing side effects and exceptions.Suggested fix: Reuse the existing length-hint implementation on the original iterable with default 32. Add this before the loop in finding 2. This P3 can be handled here or in a follow-up.
iterator = generic_getiter(tx, iterable) + variables.BuiltinVariable(operator.length_hint).call_function( + tx, [iterable, ConstantVariable.create(32)], {} + ) while True:
|
Thanks @williamwen42! The 30 red jobs all failed at checkout (/__w/pytorch/pytorch/.git: Permission denied) before building, so they look like a runner issue. Could you re-run the failed jobs? I'll send @pytorchbot merge once they're green. |
|
@pytorchbot rebase |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
ByteArrayVariable had no append/extend, so calling either on a bytearray graph broke. This adds both to tp_methods. Both follow the existing in-place ops on ByteArrayVariable (+=, *=, %=): build the result in a new buffer, record the mutation with side_effects, and swap self.data. For a bytearray that came in as an input, the existing bytearray_mutation replay handler writes the result back with old[:] = new after the graph, so tracing never mutates the live object. append runs the item through PyNumber_Index (so __index__ objects work) and lets the real bytearray.append raise the range error. extend takes constants (bytes, bytearray, str, lists of ints, ...) directly and lets the real bytearray.extend produce CPython's errors, including the str special case. Other iterables (generators, map, lists holding __index__ objects) are unpacked item by item through PyNumber_Index; a non-iterable raises "can't extend bytearray with X". If an item is not a constant after PyNumber_Index the call is declined as before. Removes 4 expected-failure markers that now pass: CPython313-test_bytes-ByteArrayTest.test_append, test_extend, test_exhausted_iterator, and CPython313-test_builtin-BuiltinTest.test_bytearray_extend_error. Part of pytorch#192126 Test Plan: ``` python test/dynamo/test_bytearray.py PYTORCH_TEST_WITH_DYNAMO=1 python test/cpython/v3_13/test_bytes.py PYTORCH_TEST_WITH_DYNAMO=1 python test/cpython/v3_13/test_builtin.py ``` New tests: test_append, test_extend and test_append_extend_arg_mutation fail on main and pass with this change; test_append_graph_break covers write-back across a graph break. I also ran every test/cpython/v3_13 file under PYTORCH_TEST_WITH_DYNAMO=1 before and after; the only new unexpected successes were the 4 markers removed here. Fix written with AI assistance (Claude), reviewed and tested by me.
Python 3.12 raises a generic TypeError for bytearray.extend(str); the "expected iterable of integers" wording is 3.13+. Check only the type. Fix written with AI assistance (Claude), reviewed and tested by me.
… first bad byte Only literal constants take the extend fast path now. Iterator variables are also python constants, but reading one returns a copy, so the traced iterator was never advanced (next(it) after extend still returned the first item). The 0..255 range check now runs per item while the iterable is consumed, so a bad byte stops iteration where CPython does instead of after draining it. test_append_graph_break now counts compiled frames, so it fails if append or extend graph-break instead of being traced.
Addresses the review on pytorch#199675. bytearray.extend handed the argument to unpack_and_apply_fn, which differs from CPython's bytearray_extend in three ways: - A bytes subclass that overrides __iter__ was iterated. CPython checks for a buffer first and never calls __iter__, so the result differed. Raw-buffer access is not modeled here, so bytes subclasses now decline before the iterable branch. - unpack_and_apply_fn drains list and tuple iterators before the per-item callback runs, so after a ValueError for an out-of-range byte the rest of the iterator was gone. It also snapshots a list, missing items appended by __index__. The loop now uses generic_getiter and pyiter_next and validates each item before reading the next one. - CPython asks the argument for a length hint (default 32) after getting the iterator and before reading any item. That call was skipped, losing its side effects and exceptions. It is now made through operator.length_hint. test_append_graph_break now has a tensor op on both sides of each mutation and checks op_count, so it fails if append or extend graph-break instead of being traced. Before, frame_count == 2 also held when extend fell back in the resumed frame. Test Plan: ``` python test/dynamo/test_bytearray.py PYTORCH_TEST_WITH_DYNAMO=1 python test/cpython/v3_13/test_bytes.py ByteArrayTest ``` test_bytearray.py: 61 passed on Python 3.13. The four new extend tests fail without this change. The CPython ByteArrayTest results are the same before and after this change. Written with AI assistance (Claude).
CPython's bytearray_extend takes the buffer path for any buffer exporter, so a bytearray subclass that overrides __iter__ is extended from its buffer, like a bytes subclass. The decline added in the previous commit only covered bytes: a bytearray subclass with Python __iter__ and __len__ becomes a UserDefinedObjectVariable and was iterated, giving b'qt' instead of b'qab'. Decline bytes and bytearray subclasses alike. A plain bytearray argument is a ByteArrayVariable and still takes the traced path. Test Plan: ``` python test/dynamo/test_bytearray.py ``` 61 passed on Python 3.13. test_extend_bytes_subclass_reads_the_buffer now also runs a bytearray subclass, which fails without this change. Written with AI assistance (Claude).
|
Successfully rebased |
c204f1d to
cabbb56
Compare
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 jobs have failed, first few of them are: pull / linux-jammy-aarch64-py3.11-clang21 / build Details for Dev Infra teamRaised by workflow job |
|
@pytorchbot merge -i |
Merge startedYour change will be merged while ignoring the following 1 checks: pull / linux-jammy-aarch64-py3.11-clang21 / build Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Dr.CI answers HTTP 400 for any exception it hits while recomputing. The three retries ran back to back in the same second, so a short upstream blip failed all of them; trymerge then fell back to the Dr.CI check run summary, which deliberately drops AI_NOT_RELATED, and blocked merges on failures the AI CI Advisor had cleared (#199675, #201134). Retry after 5, 15 and 30 seconds instead, and print the first 500 bytes of the response body, where Dr.CI names the exception.
✴️ iz2: Dr.CI's `/api/drci/drci` answers HTTP 400 for any exception it hits while recomputing. trymerge retried three times back to back, so a short upstream blip failed every attempt. It then fell back to the Dr.CI check run summary, which deliberately drops `AI_NOT_RELATED`, and blocked the merge on failures the AI CI Advisor had cleared. This happened on #199675 and #201134; both were later merged with `-i`. This change, in `get_drci_classifications`: - makes up to 4 attempts. After HTTP 400, 408, 429 or 5xx, a temporary DNS failure, a kernel connect timeout, or a connection dropped or cut off mid-response, it pauses 5, 15 and 30 seconds first. Refused connections, unknown hosts, no route (as on internal runners) and unexpected exceptions retry immediately; - does not retry a call that sat silent for the whole socket timeout, an error body that stalls, or a 504 (HUD ran out its 900 s `maxDuration`); - gives Dr.CI calls a 960 s socket timeout, so a stalled response can no longer hang the merge. `gh_fetch_url` gains an optional `timeout`; other callers are unchanged; - logs `Attempt N of TOTAL` with the first 500 bytes of the error body, where Dr.CI names the exception, next to the existing `Error fetching ... HTTP Error 400` line. Each call keeps its own retry state; during a full Dr.CI outage every failed call pays its ~50 s of pauses, and the merge loop's next iteration (`MERGE_RETRY_INTERVAL_SECONDS`, 5 min) tries again. Returning `None` on failure, the `From Dr.CI API: null` diagnostic and the stale-summary guard are unchanged. In the last two days of merge jobs (72), 8 Dr.CI calls in 6 jobs hit a 400. Back-to-back retries recovered 7 of them. Test plan: `python -m unittest test_trymerge` in `.github/scripts`, including new `TestDrciRetries` and `TestFetchUrlTimeout`. Pull Request resolved: #202124 Approved by: https://github.com/huydhn
…#202124) ✴️ iz2: Dr.CI's `/api/drci/drci` answers HTTP 400 for any exception it hits while recomputing. trymerge retried three times back to back, so a short upstream blip failed every attempt. It then fell back to the Dr.CI check run summary, which deliberately drops `AI_NOT_RELATED`, and blocked the merge on failures the AI CI Advisor had cleared. This happened on pytorch#199675 and pytorch#201134; both were later merged with `-i`. This change, in `get_drci_classifications`: - makes up to 4 attempts. After HTTP 400, 408, 429 or 5xx, a temporary DNS failure, a kernel connect timeout, or a connection dropped or cut off mid-response, it pauses 5, 15 and 30 seconds first. Refused connections, unknown hosts, no route (as on internal runners) and unexpected exceptions retry immediately; - does not retry a call that sat silent for the whole socket timeout, an error body that stalls, or a 504 (HUD ran out its 900 s `maxDuration`); - gives Dr.CI calls a 960 s socket timeout, so a stalled response can no longer hang the merge. `gh_fetch_url` gains an optional `timeout`; other callers are unchanged; - logs `Attempt N of TOTAL` with the first 500 bytes of the error body, where Dr.CI names the exception, next to the existing `Error fetching ... HTTP Error 400` line. Each call keeps its own retry state; during a full Dr.CI outage every failed call pays its ~50 s of pauses, and the merge loop's next iteration (`MERGE_RETRY_INTERVAL_SECONDS`, 5 min) tries again. Returning `None` on failure, the `From Dr.CI API: null` diagnostic and the stale-summary guard are unchanged. In the last two days of merge jobs (72), 8 Dr.CI calls in 6 jobs hit a 400. Back-to-back retries recovered 7 of them. Test plan: `python -m unittest test_trymerge` in `.github/scripts`, including new `TestDrciRetries` and `TestFetchUrlTimeout`. Pull Request resolved: pytorch#202124 Approved by: https://github.com/huydhn
…#202124) ✴️ iz2: Dr.CI's `/api/drci/drci` answers HTTP 400 for any exception it hits while recomputing. trymerge retried three times back to back, so a short upstream blip failed every attempt. It then fell back to the Dr.CI check run summary, which deliberately drops `AI_NOT_RELATED`, and blocked the merge on failures the AI CI Advisor had cleared. This happened on pytorch#199675 and pytorch#201134; both were later merged with `-i`. This change, in `get_drci_classifications`: - makes up to 4 attempts. After HTTP 400, 408, 429 or 5xx, a temporary DNS failure, a kernel connect timeout, or a connection dropped or cut off mid-response, it pauses 5, 15 and 30 seconds first. Refused connections, unknown hosts, no route (as on internal runners) and unexpected exceptions retry immediately; - does not retry a call that sat silent for the whole socket timeout, an error body that stalls, or a 504 (HUD ran out its 900 s `maxDuration`); - gives Dr.CI calls a 960 s socket timeout, so a stalled response can no longer hang the merge. `gh_fetch_url` gains an optional `timeout`; other callers are unchanged; - logs `Attempt N of TOTAL` with the first 500 bytes of the error body, where Dr.CI names the exception, next to the existing `Error fetching ... HTTP Error 400` line. Each call keeps its own retry state; during a full Dr.CI outage every failed call pays its ~50 s of pauses, and the merge loop's next iteration (`MERGE_RETRY_INTERVAL_SECONDS`, 5 min) tries again. Returning `None` on failure, the `From Dr.CI API: null` diagnostic and the stale-summary guard are unchanged. In the last two days of merge jobs (72), 8 Dr.CI calls in 6 jobs hit a 400. Back-to-back retries recovered 7 of them. Test plan: `python -m unittest test_trymerge` in `.github/scripts`, including new `TestDrciRetries` and `TestFetchUrlTimeout`. Pull Request resolved: pytorch#202124 Approved by: https://github.com/huydhn
ByteArrayVariable had no append/extend, so calling either on a bytearray
graph broke. This adds both to tp_methods.
Both follow the existing in-place ops on ByteArrayVariable (+=, *=, %=):
build the result in a new buffer, record the mutation with side_effects,
and swap self.data. For a bytearray that came in as an input, the existing
bytearray_mutation replay handler writes the result back with old[:] = new
after the graph, so tracing never mutates the live object.
append runs the item through PyNumber_Index (so index objects work)
and lets the real bytearray.append raise the range error. extend takes
constants (bytes, bytearray, str, lists of ints, ...) directly and lets
the real bytearray.extend produce CPython's errors, including the str
special case. Other iterables (generators, map, lists holding index
objects) are unpacked item by item through PyNumber_Index; a non-iterable
raises "can't extend bytearray with X". If an item is not a constant after
PyNumber_Index the call is declined as before.
Removes 4 expected-failure markers that now pass:
CPython313-test_bytes-ByteArrayTest.test_append, test_extend,
test_exhausted_iterator, and CPython313-test_builtin-BuiltinTest.test_bytearray_extend_error.
Part of #192126
Test Plan:
New tests: test_append, test_extend and test_append_extend_arg_mutation fail
on main and pass with this change; test_append_graph_break covers write-back
across a graph break. I also ran every test/cpython/v3_13 file under
PYTORCH_TEST_WITH_DYNAMO=1 before and after; the only new unexpected
successes were the 4 markers removed here.
Fix written with AI assistance (Claude), reviewed and tested by me.
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @kadeng @chauhang @amjames @jataylo @azahed98