Skip to content

Commit a36e050

Browse files
mushuiyu886jalehman
mushuiyu886
and
jalehman
authored
fix #88009: [Feature]: batched memory embedding should batch over files (#89138)
Merged via squash. Prepared head SHA: 66d362a Co-authored-by: mushuiyu886 <[email protected]> Co-authored-by: jalehman <[email protected]> Reviewed-by: @jalehman
1 parent 2f02bbc commit a36e050

13 files changed

Lines changed: 1560 additions & 234 deletions

File tree

‎docs/plugins/sdk-overview.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,16 @@ can consume this generic provider surface. The older
113113
`contracts.memoryEmbeddingProviders` seam is deprecated compatibility while
114114
existing memory-specific providers migrate.
115115

116+
Memory-specific providers that still expose a runtime `batchEmbed(...)` stay on
117+
the existing per-file batching contract unless their runtime explicitly sets
118+
`sourceWideBatchEmbed: true`. That opt-in lets the memory host submit chunks from
119+
multiple dirty memory files and enabled sources in one `batchEmbed(...)` call up
120+
to the host batch limits. Batch adapters that upload JSONL request files must
121+
split provider jobs before their upload-size cap as well as their request-count
122+
cap. The provider must return one embedding per input chunk in the same order as
123+
`batch.chunks`; omit the flag when the provider expects file-local batches or
124+
cannot preserve input ordering across a larger source-wide job.
125+
116126
### Tools and commands
117127

118128
Use [`defineToolPlugin`](/plugins/tool-plugins) for simple tool-only plugins

0 commit comments

Comments
 (0)