fix(metrics): declare output_type for AgentGoalAccuracyWithoutReference - #3015

Issue Link / Problem Description

• Fixes AgentGoalAccuracyWithoutReference never sets output_type, so metric.train() instruction optimization aborts with ValueError #3014

AgentGoalAccuracyWithoutReference never declares output_type, so it keeps the base-class default output_type: t.Optional[MetricOutputType] = None (src/ragas/metrics/base.py:167). BaseMetric._optimize_instruction reads that field to pick the loss function and raises before the optimizer is even constructed (src/ragas/metrics/base.py:211-213), so the public metric.train(path, instruction_config=InstructionConfig(llm=...)) always aborts with:

ValueError: Output type for metric 'agent_goal_accuracy' is not defined. Please set the output type in the metric or in the instruction config.

Instruction/prompt optimization is unreachable for this metric regardless of dataset size or LLM.

Its sibling AgentGoalAccuracyWithReference, defined in the same module and scoring the same Literal["0", "1"] verdict (CompareOutcomeOutput), does declare output_type: t.Optional[MetricOutputType] = MetricOutputType.BINARY at src/ragas/metrics/_goal_accuracy.py:114. Commit 9bd1402 (PR #1722), which introduced output_type so the loss for optimization could be derived, added that line to AgentGoalAccuracyWithReference only and skipped the near-identical class sitting directly below it in the same file. The value is enforced independently downstream as well: src/ragas/optimizers/genetic.py:549 raises ValueError("No output type provided for the metric.").

Changes Made

• src/ragas/metrics/_goal_accuracy.py: add output_type: t.Optional[MetricOutputType] = MetricOutputType.BINARY to the AgentGoalAccuracyWithoutReference dataclass body, directly after _required_columns, mirroring line 114 of the sibling class. MetricOutputType is already imported in the module. This is the only production change.
• tests/unit/test_goal_accuracy.py (new): regression test covering both the declared value and the train() path that consumes it.

Testing

How to Test

• Automated tests added/updated
• Manual testing steps:

pytest tests/unit/test_goal_accuracy.py

On an unpatched tree (source reverted to main, new test kept):

FAILED tests/unit/test_goal_accuracy.py::test_goal_accuracy_metrics_declare_output_type[AgentGoalAccuracyWithoutReference]
FAILED tests/unit/test_goal_accuracy.py::test_instruction_training_derives_loss_from_output_type
2 failed, 1 passed, 4 warnings in 0.12s

with src/ragas/metrics/base.py:212: ValueError: Output type for metric 'agent_goal_accuracy' is not defined. Please set the output type in the metric or in the instruction config.

On the patched tree:

tests/unit/test_goal_accuracy.py ...                                     [100%]
3 passed, 4 warnings in 0.04s

The second test drives AgentGoalAccuracyWithoutReference(llm=...).train(path, instruction_config=InstructionConfig(llm=...)) with an optimizer stub that records the loss it is handed, and asserts it is a BinaryMetricLoss — i.e. the same run that previously raised now reaches the optimizer.

Full unit suite (pytest tests/unit --ignore=tests/unit/integrations): 2 failed, 743 passed, 62 skipped, 8 errors. The identical failure/error set occurs on unmodified main (missing optional sacrebleu / haystack dependencies and test_experiment.py git fixtures); the unpatched run differs only by the two new test failures. No regressions introduced.

Lint and type checks: ruff format --check and ruff check pass on both changed files (the repo-wide ruff format --check reports 88 pre-existing reformats in files this PR does not touch); pyright -p pyproject.toml reports 0 errors for src/ragas/metrics/_goal_accuracy.py.

References

• Issue: AgentGoalAccuracyWithoutReference never sets output_type, so metric.train() instruction optimization aborts with ValueError #3014
• Related: PR feat: add output type to metrics #1722 / commit 9bd1402 ("feat: add output type to metrics") — added output_type to the sibling class only
• Downstream enforcement of the same field: src/ragas/optimizers/genetic.py:549

Screenshots/Examples (if applicable)

N/A — one-line field declaration.