Skip to content

tool_with(): if/elif chain silently drops viewer/model_input when combined with parallel (or with each other) #5896

Open

Description

@BlueX888

What happened, and what did you expect?

tool_with() silently drops viewer and/or model_input when they are passed together with parallel (or with each other). Only the first matching argument of an elif chain is applied; the dropped attributes keep their previous value (None) with no error or warning, so a custom tool-call view or model-input presenter never takes effect:

  • tool_with(tool, parallel=True, viewer=viewer) applies only parallel and silently discards viewer
  • tool_with(tool, viewer=viewer, model_input=model_input) applies only viewer and discards model_input
  • with all three kwargs, only parallel is applied

I expected each of parallel, viewer and model_input to be applied independently. The docstring of tool_with() ("Tool with modifications to various attributes", src/inspect_ai/tool/_tool_with.py:23-42) documents each argument separately with no exclusivity language, and the sibling code paths treat the three attributes as independent: ToolDef.__init__ assigns self.parallel, self.viewer and self.model_input in three separate statements (src/inspect_ai/tool/_tool_def.py:94-96), and the @tool decorator writes TOOL_PARALLEL, TOOL_VIEWER and TOOL_MODEL_INPUT into one metadata dict unconditionally (src/inspect_ai/tool/_tool.py:264-273).

This is reachable from the public API: tool_with is exported from inspect_ai.tool (src/inspect_ai/tool/__init__.py:46,86). For example, the web browser tools are themselves built as tool_with(tool, viewer=web_at_viewer) (src/inspect_ai/tool/_tools/_web_browser/_web_browser.py:182), so adding parallel=True to such a call silently loses the custom view. The attributes are consumed at eval time by tool_def_fields (src/inspect_ai/tool/_tool_def.py:267-276) for transcript rendering and model-input playback.

Steps to reproduce

from inspect_ai.tool import tool
from inspect_ai.tool._tool import TOOL_MODEL_INPUT, TOOL_PARALLEL, TOOL_VIEWER
from inspect_ai.tool._tool_call import ToolCallView
from inspect_ai.tool._tool_def import tool_registry_info
from inspect_ai.tool._tool_with import tool_with


@tool
def demo() -> None:
    async def execute(x: int) -> int:
        """Add one.

        Args:
          x: Input number.
        """
        return x + 1

    return execute


def viewer(call) -> ToolCallView:
    return ToolCallView()


def model_input(index, total, content, hints):
    return content


def meta(t):
    reg = tool_registry_info(t)
    return {
        "parallel": reg.parallel,
        "viewer": reg.viewer,
        "model_input": reg.model_input,
    }


t = demo()
tool_with(t, parallel=True, viewer=viewer)
print("parallel+viewer ->", meta(t))

t2 = demo()
tool_with(t2, viewer=viewer, model_input=model_input)
print("viewer+model_input ->", meta(t2))

t3 = demo()
tool_with(t3, parallel=True, viewer=viewer, model_input=model_input)
print("all three ->", meta(t3))

t4 = demo()
tool_with(t4, viewer=viewer)
print("viewer alone ->", meta(t4))

t5 = demo()
tool_with(t5, model_input=model_input)
print("model_input alone ->", meta(t5))

t6 = demo()
tool_with(t6, parallel=True)
print("parallel alone ->", meta(t6))

Actual output on this commit (function addresses vary per run; reproduced identically in three separate runs):

parallel+viewer -> {'parallel': True, 'viewer': None, 'model_input': None}
viewer+model_input -> {'parallel': False, 'viewer': <function viewer at 0x108e9bd80>, 'model_input': None}
all three -> {'parallel': True, 'viewer': None, 'model_input': None}
viewer alone -> {'parallel': False, 'viewer': <function viewer at 0x108e9bd80>, 'model_input': None}
model_input alone -> {'parallel': False, 'viewer': None, 'model_input': <function model_input at 0x105c8e020>}
parallel alone -> {'parallel': True, 'viewer': None, 'model_input': None}

Each argument alone works; combining them applies only the first and silently drops the rest.

Root cause: src/inspect_ai/tool/_tool_with.py:62-67 resolves the three attributes with an elif chain (if parallel is not None: ... elif viewer is not None: ... elif model_input is not None: ...), so only the first non-None argument is written into the registry metadata; the others never reach the metadata the eval-time consumer reads.

I'd be happy to open a PR that resolves each attribute with an independent guard, mirroring ToolDef.__init__ and the @tool decorator, plus a regression test combining two of the kwargs.

Searched for existing reports and competing work: a full-text search for tool_with over issues finds only #1471 (basic_agent infinite tool init) and #678 (feature request to expose private elements), both closed and unrelated; no open PR touches src/inspect_ai/tool/_tool_with.py (open PR #5778 concerns tool approval and failing viewers but does not modify this file).

Version and environment

inspect_ai at commit 65447aa49aaae4aa36755ea4b9fe7fa798bed518 (main, after release 0.3.281), Python 3.12.14, macOS 26.6.2 (arm64).

Note: this report was prepared with AI assistance — an AI coding agent ran the reproduction and verification steps described above; happy to be assigned or to open the PR.

Activity

  1. BlueX888 commented on Oct 16, 2026

    @BlueX888
    Author

    Reproduced this. Root cause: src/inspect_ai/tool/_tool_with.py:62-67 resolves parallel, viewer and model_input through an if/elif chain, so only the first non-None argument is written into the tool's registry metadata; the others keep their previous value (None) with no error or warning.

    Minimal reproduction:

    from inspect_ai.tool import tool
    from inspect_ai.tool._tool import TOOL_MODEL_INPUT, TOOL_PARALLEL, TOOL_VIEWER
    from inspect_ai.tool._tool_call import ToolCallView
    from inspect_ai.tool._tool_def import tool_registry_info
    from inspect_ai.tool._tool_with import tool_with
    
    
    @tool
    def demo() -> None:
        async def execute(x: int) -> int:
            """Add one.
    
            Args:
              x: Input number.
            """
            return x + 1
    
        return execute
    
    
    def viewer(call) -> ToolCallView:
        return ToolCallView()
    
    
    def model_input(index, total, content, hints):
        return content
    
    
    def meta(t):
        reg = tool_registry_info(t)
        return {
            "parallel": reg.parallel,
            "viewer": reg.viewer,
            "model_input": reg.model_input,
        }
    
    
    t = demo()
    tool_with(t, parallel=True, viewer=viewer)
    print("parallel+viewer ->", meta(t))
    
    t2 = demo()
    tool_with(t2, viewer=viewer, model_input=model_input)
    print("viewer+model_input ->", meta(t2))
    
    t3 = demo()
    tool_with(t3, parallel=True, viewer=viewer, model_input=model_input)
    print("all three ->", meta(t3))
    
    t4 = demo()
    tool_with(t4, viewer=viewer)
    print("viewer alone ->", meta(t4))
    
    t5 = demo()
    tool_with(t5, model_input=model_input)
    print("model_input alone ->", meta(t5))
    
    t6 = demo()
    tool_with(t6, parallel=True)
    print("parallel alone ->", meta(t6))
    

    Observed output:

    parallel+viewer -> {'parallel': True, 'viewer': None, 'model_input': None}
    viewer+model_input -> {'parallel': False, 'viewer': <function viewer at 0x108e9bd80>, 'model_input': None}
    all three -> {'parallel': True, 'viewer': None, 'model_input': None}
    viewer alone -> {'parallel': False, 'viewer': <function viewer at 0x108e9bd80>, 'model_input': None}
    model_input alone -> {'parallel': False, 'viewer': None, 'model_input': <function model_input at 0x105c8e020>}
    parallel alone -> {'parallel': True, 'viewer': None, 'model_input': None}
    

    Fix is ready on BlueX888:fix/prep-tool-with-elif-drops-attrs (commit a6289d0a). Replaces the elif chain with independent if guards for each attribute, matching how the @tool decorator and ToolDef.init already set all three, plus regression tests for the combined cases.
    Verified with: python -m pytest tests/tools/test_tool_with.py (6 passed: 4 new regression tests for the combined cases + the 2 pre-existing tests)

    Happy to open the PR once you confirm the approach or assign this to me.

  2. added
    acceptedIssues: maintainer-approved for external PR — the gate key
    bugSomething isn't working
    on Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    acceptedIssues: maintainer-approved for external PR — the gate keybugSomething isn't workinggood first issueGood for newcomers

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions