AWS Bedrock: non-tool Amazon Nova path sends Converse but parses with the invoke_model parser, so generate_response returns "Error parsing response" #7360

Component

Python SDK

Summary

AWSBedrockLLM._generate_standard returns the literal string "Error parsing response" for every successful Amazon Nova text call that does not use tools.

When provider == "amazon" and "nova" is in the model id and no tools are supplied, the function sends the request through the Converse API (self.client.converse(...), mem0/llms/aws_bedrock.py:626-630) and then hands that reply to the legacy invoke_model parser:

return self._parse_response(response)   # mem0/llms/aws_bedrock.py:631

_parse_response only understands the invoke_model shape. Its text branch opens with

response_body = response.get("body").read().decode()   # mem0/llms/aws_bedrock.py:394

A Converse reply is an already-parsed dict with no "body" key, so response.get("body") is None and None.read() raises AttributeError. The bare except at line 431 swallows it:

except Exception as e:
    logger.warning(f"Could not parse response: {e}")
    return "Error parsing response"          # mem0/llms/aws_bedrock.py:433

The model's text is discarded and the caller cannot distinguish a successful call from a failure. The Nova sub-branch inside _parse_response (lines 402-407, looking for "content"/"completion" in a JSON body) is unreachable from this call site, which is what confirms the routing mistake rather than a missing body format.

Steps to Reproduce

No AWS credentials or network access are needed — the Bedrock client is mocked, and the mocked return value is the exact dict shape the Converse API returns.

import logging
from unittest.mock import MagicMock, patch

logging.disable(logging.CRITICAL)

with patch("mem0.llms.aws_bedrock.boto3") as mock_b3:
    runtime, bedrock = MagicMock(), MagicMock()
    bedrock.list_foundation_models.return_value = {"modelSummaries": []}
    mock_b3.client.side_effect = lambda service, **kw: runtime if service == "bedrock-runtime" else bedrock

    from mem0.configs.llms.aws_bedrock import AWSBedrockConfig
    from mem0.llms.aws_bedrock import AWSBedrockLLM

    messages = [{"role": "system", "content": "extract"}, {"role": "user", "content": "hi"}]
    # exact shape boto3 client.converse() returns (a parsed dict, no "body" stream)
    converse_reply = {"output": {"message": {"role": "assistant", "content": [{"text": "The sky is blue."}]}},
                      "stopReason": "end_turn"}

    runtime.converse.return_value = converse_reply
    nova = AWSBedrockLLM(AWSBedrockConfig(model="amazon.nova-3-mini-20241119-v1:0"))
    print("nova generate_response ->", repr(nova.generate_response(messages, response_format={"type": "json_object"})))
    print("   converse called:", runtime.converse.called, "| invoke_model called:", runtime.invoke_model.called)

    runtime.reset_mock()
    runtime.converse.return_value = converse_reply
    anthropic = AWSBedrockLLM(AWSBedrockConfig(model="anthropic.claude-3-5-sonnet-20240620-v1:0"))
    print("anthropic generate_response ->", repr(anthropic.generate_response(messages, response_format={"type": "json_object"})))

Expected Behavior

generate_response() should return the model's text ("The sky is blue.") for a Nova non-tool call, exactly as it does for the Anthropic control case in the same script, which receives the identical Converse reply dict.

Actual Behavior

nova generate_response -> 'Error parsing response'
   converse called: True | invoke_model called: False
anthropic generate_response -> 'The sky is blue.'

No exception reaches the caller and no traceback is printed; the only signal is logger.warning("Could not parse response: 'NoneType' object has no attribute 'read'"), which is why this is easy to miss in production.

Environment

• mem0 version: 2.0.20 (pyproject.toml), mem0ai Python SDK
• Commit: c7ee362aff94a369af70f13f2b4f853f6793ff4c
• Python version: 3.12.14
• OS: macOS 26.6.2 (arm64)
• Provider / model: aws_bedrock, amazon.nova-3-mini-20241119-v1:0 (also reproduced on amazon.nova-lite-v1:0 and amazon.nova-pro-v1:0)

How You Verified This

What I Ran

The script in Steps to Reproduce, saved as repro_nova_parse.py, run against a checkout of mem0ai/mem0 pinned at c7ee362aff94a369af70f13f2b4f853f6793ff4c:

PYTHONPATH=<mem0 checkout> python3 repro_nova_parse.py

The bug was reproduced a second time by an independent checker using a separate harness that also swept amazon.nova-lite-v1:0, amazon.nova-pro-v1:0, the Nova-plus-tools path, and a direct _parse_response call on a genuine invoke_model body.

What I Saw

Run 1 (this report's script, verbatim):

nova generate_response -> 'Error parsing response'
   converse called: True | invoke_model called: False
anthropic generate_response -> 'The sky is blue.'

Run 2 (independent harness, verbatim):

nova[amazon.nova-lite-v1:0] provider='amazon' supports_tools=True
 generate_response -> 'Error parsing response'
 converse=True invoke_model=False
nova[amazon.nova-pro-v1:0] -> 'Error parsing response'  (same for amazon.nova-3-mini-...)
anthropic (same Converse reply) -> '{"facts": ["sky is blue"]}'
nova+tools -> {'tool_calls': []}   |  _parse_response(invoke_model body) -> 'genuine invoke_model text'

Run 3 (earlier pass, verbatim):

provider           : amazon
converse called    : True
invoke_model called: False
nova got           : 'Error parsing response'
nova expected      : 'The sky is blue.'
anthropic got      : 'The sky is blue.'
nova (format) got  : 'Error parsing response'

With the warning logger enabled, the swallowed exception is visible as Could not parse response: 'NoneType' object has no attribute 'read'.

Why This Is a Bug

The sibling Converse branches in the same function parse the identical response object correctly. Three lines below the Nova branch, the MiniMax branch (mem0/llms/aws_bedrock.py:617-621) reads:

response = self.client.converse(**converse_params)
for block in response["output"]["message"]["content"]:
    if "text" in block:
        return block["text"]
return ""

and the Anthropic branch (mem0/llms/aws_bedrock.py:583-588) does return response['output']['message']['content'][0]['text']. _generate_with_tools also reads response["output"]["message"]["content"] for Converse replies (mem0/llms/aws_bedrock.py:380-388).

The repo's own test fixture for a Converse reply has exactly that shape — tests/llms/test_aws_bedrock.py:36-38:

def _converse_response(text: str = "ok") -> dict:
    """Minimal Converse API response dict."""
    return {"output": {"message": {"content": [{"text": text}]}}}

i.e. a parsed dict, never a stream under "body". The test file's own section header at tests/llms/test_aws_bedrock.py:484 labels _parse_response as "legacy InvokeModel provider-specific parsing", and botocore's ConverseResponse members (bedrock-runtime/2023-09-30) are [output, stopReason, usage, metrics, additionalModelResponseFields, trace, performanceConfig, serviceTier] — there is no body.

Reachability: this is the normal production path with no tools. Memory.add() (mem0/memory/main.py:956-962) calls self.llm.generate_response(messages=[system, user], response_format={"type": "json_object"}) with tools unset; LlmFactory.create("aws_bedrock", config) returns AWSBedrockLLM, whose generate_response with tools=None dispatches to _generate_standard, takes the amazon + nova branch at line 623, and reaches line 631. Downstream, add() strips and parses that string, json.loads fails, extract_json leaves it unchanged, and the result degrades to extracted_memories = [] — a silent no-op where nothing is stored, with no error surfaced to the user.

Root cause — mem0/llms/aws_bedrock.py:631 in AWSBedrockLLM._generate_standard: the Nova non-tool branch issues a Converse call but delegates to _parse_response, which only handles invoke_model replies and raises AttributeError on the missing "body" key (line 394); the bare except at lines 431-433 converts that into the sentinel string.

Suggested approach (not applied here — happy to prepare the PR once this is accepted): parse the Converse reply in the Nova branch the way the MiniMax/Anthropic branches do, e.g. for block in response["output"]["message"]["content"]: if "text" in block: return block["text"] with a return "" fallback; alternatively have _parse_response detect a Converse dict ("output" in response) and read it before the body-stream branch. A regression test asserting the returned value for a Nova non-tool call would close the gap: the existing Nova tests (tests/llms/test_aws_bedrock.py:241-270, 367-388) only assert the request kwargs (modelId, inferenceConfig) and never the return value.

What I Ruled Out

• Missing "body" in my mock, not a code bug. The Anthropic control case in the same script receives the byte-identical Converse dict and returns 'The sky is blue.', so the difference is the branch taken, not the fixture. The fixture also matches the repo's own _converse_response() helper and botocore's modelled response members.
• A tools-path problem. _generate_standard is only reached when tools is falsy (generate_response routes to _generate_with_tools otherwise), and add() calls without tools. The Nova-plus-tools path returns {'tool_calls': []} and is unaffected.
• A region/credential issue. The client is mocked; no AWS call is made.
• The amazon branch of _parse_response being simply unreachable. _parse_response on a genuine invoke_model body still returns real text, including for Nova (nova (format) got: 'Error parsing response' is the Converse path; the direct invoke_model call returns text), so the bug is specific to this call site.

AI Assistance

AI helped me find it, and I reproduced it myself afterwards.

The audit that surfaced this candidate was AI-assisted, and the write-up was drafted with AI help. The reproduction above was then run against commit c7ee362aff94a369af70f13f2b4f853f6793ff4c (twice independently, output pasted verbatim above) before filing.

Related

• AWS Bedrock LLM returns "Error parsing response" string on parse failure instead of raising #6548 / fix(llms/aws_bedrock): raise on parse failure instead of returning a sentinel #6551 (open) — _parse_response returns the sentinel string instead of raising. That is about the sentinel design; this issue is about a Converse response being routed into the invoke_model parser in the first place. Fixing AWS Bedrock LLM returns "Error parsing response" string on parse failure instead of raising #6548 would turn this into a raised exception rather than a wrong string, but the Nova branch would still fail.
• [Bug] AWS Bedrock: legacy amazon (Titan) responses always parse to '' — _parse_response reads a 'completion' field Titan never returns #7336 (open) — the sibling legacy Amazon/Titan branch of the same function returns '' for a different reason (reads completion, which Titan does not return). Different branch, different line, different symptom.