thank you for the fix. I have a question: should we call context.Flush() after each prepack (or after the initialization?)

thank you for the fix. I have a question: should we call context.Flush() after each prepack (or after the initialization?)

Good point. I definitely don't want to call context.Flush() after the initialization which is not good for memory. To not impact the original initializer, I add a new one specific for prepack which will follow the normal buffer manager to flush when pending patches reach to max_num_pending_dispatches_. What do you think?

thank you for the fix. I have a question: should we call context.Flush() after each prepack (or after the initialization?)

Good point. I definitely don't want to call context.Flush() after the initialization which is not good for memory. To not impact the original initializer, I add a new one specific for prepack which will follow the normal buffer manager to flush when pending patches reach to max_num_pending_dispatches_. What do you think?

Sorry—this approach may still have issues. With this change, I can’t properly control the release of the original weights. I’ll think through this more carefully and send an updated proposal. Thanks.

thank you for the fix. I have a question: should we call context.Flush() after each prepack (or after the initialization?)

Good point. I definitely don't want to call context.Flush() after the initialization which is not good for memory. To not impact the original initializer, I add a new one specific for prepack which will follow the normal buffer manager to flush when pending patches reach to max_num_pending_dispatches_. What do you think?

Sorry—this approach may still have issues. With this change, I can’t properly control the release of the original weights. I’ll think through this more carefully and send an updated proposal. Thanks.

Restore to the original one with LazyRelease plus flush after each prepack.

With this PR, the issue #27068 cannot been reproduced again. Thanks.