Repository navigation
Open
Description
Description
In the LangGraph adapter, any Agent that declares outputs fails on its first model call when the LLM is a Cohere model on OCI Generative AI (OciGenAiConfig with e.g. cohere.command-a-03-2025):
ValueError: Tool choice is not supported for Cohere models.Please remove the tool_choice parameter.
The same agent without outputs works, with or without tools. Found while investigating #224 and #227 (the structured-output path of the adapter).
Reproduction
from pyagentspec.adapters.langgraph import AgentSpecLoader
from pyagentspec.agent import Agent
from pyagentspec.llms import OciGenAiConfig
from pyagentspec.llms.ociclientconfig import OciClientConfigWithSecurityToken
from pyagentspec.property import StringProperty
llm = OciGenAiConfig(
name="oci",
model_id="cohere.command-a-03-2025",
compartment_id="<compartment_ocid>",
client_config=OciClientConfigWithSecurityToken(
name="cc",
service_endpoint="https://inference.generativeai.<region>.oci.oraclecloud.com",
auth_file_location="~/.oci/config",
auth_profile="<profile>",
),
)
agent = Agent(
name="assistant",
llm_config=llm,
system_prompt="You are a concise assistant.",
outputs=[StringProperty(title="answer")], # <- remove this line and the agent works
)
graph = AgentSpecLoader().load_component(agent) # OK
graph.invoke({"messages": [{"role": "user", "content": "What is 3 times 6?"}]},
{"configurable": {"thread_id": "1"}}) # failsObserved live with langchain-oci 0.2.6, langchain 1.3.9, langgraph 1.2.4, Python 3.12:
| agent | result |
|---|---|
| no outputs, no tools | works, answers 18 |
| a tool, no outputs | works, calls the tool |
outputs=[answer], no tools |
ValueError at first invoke |
outputs=[answer] + a tool |
ValueError at first invoke |
Traceback excerpt:
File ".../langchain/agents/factory.py", line 1364, in _get_bound_model
request.model.bind_tools(
File ".../langchain_oci/chat_models/oci_generative_ai.py", line 311, in bind_tools
kwargs["tool_choice"] = self._provider.process_tool_choice(tool_choice)
File ".../langchain_oci/chat_models/providers/cohere.py", line 721, in process_tool_choice
raise ValueError("Tool choice is not supported for Cohere models."
"Please remove the tool_choice parameter.")
Root cause
_create_react_agent_with_given_info(pyagentspec/src/pyagentspec/adapters/langgraph/_langgraphconverter.py, around theresponse_format = ToolStrategy(output_model)line) always usesToolStrategyfor agents with outputs, regardless of the provider.- LangChain's
create_agentbinds the structured-output tool withtool_choice="any"whenever aToolStrategyis active (langchain/agents/factory.py,_get_bound_model:tool_choice = "any" if structured_output_tools else request.tool_choice). - langchain-oci's
CohereProvider.process_tool_choiceraises for any non-Nonetool_choice. This is unchanged in langchain-oci 0.3.2 (the latest release) and on itsmain, so the dependency bump in fix(langgraph): structured LlmNode outputs with OciGenAiConfig require langchain-oci>=0.3.0 #250 does not change the outcome.
The failure only surfaces at the first model call, not when loading the component.
Expected behavior
Agents with declared outputs run on Cohere models, or the adapter reports a clear configuration error at load time.
Suggested fixes
- For providers that do not support
tool_choice(Cohere viaChatOCIGenAI), do not useToolStrategy; use the provider-native structured output instead.ChatOCIGenAI.with_structured_outputsupportsmethod="json_schema"for Cohere (CohereProvider.oci_json_schema_response_format), which maps to LangChain'sProviderStrategy. - Alternatively, keep
ToolStrategybut bind the structured-output tool without forcingtool_choicefor such providers (needs a LangChain-side hook or a middleware adjusting the model request). - At minimum, detect the combination at load time and raise a configuration error naming the model, the
outputs, and the workaround (declare a single string output or use another model), instead of the provider's low-level error at run time.
Metadata
Metadata
Assignees
Labels
No labels
Activity