Repository navigation
fix(core): stop prompt constructors from mutating the caller's metadata dict - #23244
Lesereingrape wants to merge 2 commits into
Conversation
chrikrah
left a comment
There was a problem hiding this comment.
@Lesereingrape I would merge the production change. The Langchain hunk works, but its test cannot pass in any environment, so nothing pins it.
test_langchain_prompt_template_leaves_the_callers_dict_alone guards on langchain_core, and LangchainPromptTemplate needs langchain plus langchain_classic through core/bridge/langchain.py. With both installed, the test then passes a langchain PromptTemplate as selector=, and the constructor calls selector.default_prompt on it. Passing it as template= is what the constructor expects.
Evidence at 116ccd9, llama-index-core from the branch on PYTHONPATH:
$ pytest tests/prompts -q
34 passed, 1 skipped # the skip is the Langchain test
# prompts/base.py from origin/main, tests kept
4 failed, 30 passed, 1 skipped
# + langchain 1.4.3, langchain-classic 1.0.8, langchain-core 1.6.6
$ pytest tests/prompts/test_metadata_ownership.py -k langchain -q
AttributeError: 'PromptTemplate' object has no attribute 'default_prompt'
$ python probe.py # LangchainPromptTemplate(template=..., metadata={"team": "rag"})
branch: caller dict: {'team': 'rag'}
origin/main: caller dict: {'team': 'rag', 'prompt_type': 'langchain'}
non-blocking: guard on pytest.importorskip("llama_index.core.bridge.langchain") and pass template=. langchain-classic does not pull in langchain, so a langchain_classic guard alone fails where only that is installed. Core's pyproject has no langchain, so this test only ever skips in CI.
@Lesereingrape, want to make that switch? I'll re-run it with langchain installed.
|
Both points taken — addressed in
Measured with llama-index-core from this branch on The old form reproduces your report. Kept as So the Langchain hunk is now pinned by a test that runs wherever the bridge is importable. |
chrikrah
left a comment
There was a problem hiding this comment.
@Lesereingrape both points land in 3d51f4b and the Langchain case runs here. That is my approval.
Measured with llama-index-core from your branch on PYTHONPATH, pytest 9.1.1. langchain 1.4.3, langchain-classic 1.0.8 and langchain-core 1.6.6 went into a separate target directory:
$ pytest tests/prompts -q # 3d51f4b
35 passed in 0.76s
$ pytest tests/prompts -q # 116ccd9, the previous head
AttributeError: 'PromptTemplate' object has no attribute 'default_prompt'
1 failed, 34 passed in 0.75s
$ pytest tests/prompts -q # cf8311c, the merge base
29 passed in 0.47s
$ pytest tests/prompts -q # prompts/base.py from cf8311c, your tests kept
5 failed, 30 passed in 0.67s
$ pytest tests/prompts/test_metadata_ownership.py -q -rs # 3d51f4b, langchain uninstalled
5 passed, 1 skipped in 0.02s
SKIPPED [1] tests/prompts/test_metadata_ownership.py:70: could not import 'llama_index.core.bridge.langchain': No module named 'langchain'
$ ruff check llama_index/core/prompts/base.py tests/prompts/test_metadata_ownership.py
All checks passed! # ruff 0.11.11, the pin in the dev group
The fifth failure in that revert run is the Langchain case. Your third production hunk has a test behind it now.
non-blocking: llama-index-core/uv.lock holds no langchain entry. CI takes the skip branch, so upstream the Langchain hunk is the one with nothing on it; the other four tests pin the two dict-literal hunks only. langchain and langchain-classic in the core dev group would buy the coverage and pull langgraph and langsmith into every dev install. That is a packaging call for core, not something to add here.
@logan-markewich you merged eleven of the changes that touched prompts/base.py. Three one-line hunks and six tests. Anything blocking it going in?
Description
PromptTemplate,ChatPromptTemplateandLangchainPromptTemplaterecord which kind ofprompt they are by writing into the
metadatadict the caller handed them:So a constructor mutates its argument. On
main:Three things fall out of that, all measured locally:
common pattern of keeping one metadata mapping and passing it to prompts and to
TextNode(metadata=...)/ an observability span / a log line;prompt_type, so the caller's mapping no longer describes any of them;prompt_typein their own metadata loses that value, andhas no way to notice because nothing raises.
The fix is to build the mapping instead of writing into the argument, one line per site:
The prompt's own
metadatacontent is identical for every input (pydantic alreadystores a copy of the dict —
prompt.metadata is metadataisFalseonmaintoo), so thisonly removes the side effect on the caller's object. Nothing that works today changes.
Fixes # (no existing issue for this; filing the fix directly)
New Package?
Version Bump?
llama-index-coreis exempt per the note in this checkbox.Type of Change
How Has This Been Tested?
llama-index-core/tests/prompts/test_metadata_ownership.py(new) asserts, for each of thethree template classes, that the caller's dict is unchanged after construction, that
prompt.metadata["prompt_type"]is still recorded, that one dict reused by two promptsstays clean, and that a caller-supplied
prompt_typevalue survives in the caller's dict.The
LangchainPromptTemplatecase isimportorskip("langchain_core")(not installed in mylocal env, so it is skipped here and runs where langchain is present).
Python 3.12.11,
llama-index-core0.14.25, offline (no network in any of these runs).Red on
main(cf8311c4, this test file against the pristineprompts/base.py):The one that passes on
mainis the control (test_default_metadata_is_still_created) — theno-argument path, which is exactly the behaviour that must not move.
Green on this branch:
Lint / types:
ruffpinned to0.11.8, the rev.pre-commit-config.yamluses. I did not run the fullllama-devtest selector (its runner is Linux-only), so the numbers above are per-packagepytest runs of the prompt-adjacent suites rather than the complete
llama-index-coresuite.Suggested Checklist
The remaining checklist items are left unticked on purpose: this patch was written and
measured by an automated agent session working for the account owner, and the owner has not
independently re-run it, so the self-review / "checked this doesn't break the public
interface" attestations belong to a human, not to me. Happy to answer anything a maintainer
wants verified, and to re-run a wider suite on request.