Repository navigation
Description
Description
Multiple LangGraph agent execution tests load successfully but do not terminate. Instead, execution repeatedly traverses the graph until LangGraph raises:
langgraph.errors.GraphRecursionError:
Recursion limit of 800 reached without hitting a stop condition.
The affected test groups are:
- Basic agent execution.
- VLLM execution.
- OpenAI execution.
- OpenAI-compatible execution.
Representative tests include:
tests/agent/validation/test_valid_configs_agent.pytests/llms/execution/test_execution_llmconfig_with_agent.py
The affected LLM configurations include:
vllmconfig_with_agent.yamlopenaiconfig_with_agent.yamlopenaicompatibleconfig_with_agent.yaml
The LangGraph CTS runtime explicitly sets:
{"configurable": {"thread_id": "1"}, "recursion_limit": 800}in langgraphruntime/runtime.py:101-103.
The logs show that the graph reaches this limit without reaching a terminal state. This occurs across multiple model-provider paths, which suggests a shared graph-construction, response-normalization, or termination issue rather than an isolated provider failure.
The LangGraph adapter builds the agent graph through _create_react_agent_with_given_info() and configures the graph with:
- The converted model.
- Converted tools.
- The Agent Spec system prompt.
- The Agent Spec output model.
- The LangGraph state schema.
- The configured checkpointer.
Relevant construction code is in _langgraphconverter.py:1143-1193.
Possible failure modes include:
- A normal assistant response is incorrectly interpreted as an incomplete continuation.
- A model response is repeatedly classified as a tool call.
- The graph routing condition does not recognize the terminal response.
- The response format or structured-response extraction prevents the stop condition.
- The provider-specific response shape is not normalized into the message format expected by the graph.
- A tool or model node returns state that causes the graph to re-enter the same node.
The Ollama execution failure should be tracked separately unless the endpoint is first made available, because its log reports connection refusal rather than recursion. OCI cases should also be handled separately because they fail during authentication/configuration setup.
Expected behavior
For a valid model endpoint and valid credentials, a basic request such as:
calculate 3x6 ?
should execute once, reach a terminal assistant response, and return a valid CTS execution status containing either:
- Non-empty
agent_messages, or - Non-empty
outputs.
The graph should not require increasing the recursion limit to hide the failure.
Relevant code
agent-spec/conformance_test_suite/runtimes/langgraphruntime/src/langgraphruntime/runtime.py:101-126agent-spec/conformance_test_suite/runtimes/langgraphruntime/src/langgraphruntime/runtime.py:127-162agent-spec/pyagentspec/src/pyagentspec/adapters/langgraph/_langgraphconverter.py:1143-1193agent-spec/pyagentspec/src/pyagentspec/adapters/langgraph/_node_execution.py:549-568agent-spec/pyagentspec/src/pyagentspec/adapters/langgraph/_node_execution.py:922-942agent-spec/conformance_test_suite/tests/llms/execution/test_execution_llmconfig_with_agent.py
Activity
fede-kamel commentedon Sep 11, 2026
I could reproduce the non-termination deterministically without any provider, and I think there are two separate causes behind the recursion errors:
1. Agents with declared outputs and no tools loop when the model answers in prose. The adapter enables
response_format=ToolStrategy(...)for any agent with outputs. With no tools, LangChain'screate_agentuses itsmodel_to_modeledge, which routes back to the model as long as nostructured_responseis in the state. A model that answers in plain text therefore loops untilGraphRecursionError. All four failing CTS configs (vllmconfig_with_agent.yaml,openaiconfig_with_agent.yaml,openaicompatibleconfig_with_agent.yaml, the basic agent) haveoutputs: [llm_output]andtools: [], and the CTS deterministic LLM server answers in prose (it only honours provider-nativeresponse_format, not the structured-output tool).Fake chat model reproduction on
main(recursion limit 12):GraphRecursionErrorafter 12 model callsmodel_to_toolsexits when there are no tool calls)GraphRecursionError(LangChain retries withhandle_errors=True)Open PR #210 addresses this part: with its head applied, the CTS-shaped case terminates in 2 calls. Real providers that honour the structured-output tool terminate in one turn (checked live against OCI Generative AI models), which is why this only shows up with the deterministic server and non-compliant models.
2. Nested
AgentNodeagents ignored the run's recursion limit.create_agentbindsrecursion_limit=9999to the agents it compiles, andAgentNodeExecutorinvoked them with the loader config only, so arecursion_limitset on the flow run never applied inside anAgentNode: withrecursion_limit=12on the run, a non-terminating nested agent made 9999 model calls before failing. #261 fixes that.Two CTS-runtime observations from the same investigation: the runtime calls
run()once with no user message before appending the prompt, which some providers reject outright (OCI returns 400 when tools are bound with no user message), and the interrupt handling is theKeyErrorof #223 (patch posted there).fix(langgraph): pass the recursion limit of the flow run on to AgentN…