Skip to content

Use --target_name as the WebSocket generator uri - #2315

Merged
jmartin-tech merged 2 commits into
NVIDIA:mainfrom
Somay-kousis:fix/websocket-target-name
Oct 16, 2026
Merged

jmartin-tech merged 2 commits into
NVIDIA:mainfrom
Somay-kousis:fix/websocket-target-name

Conversation

@Somay-kousis

Copy link
Copy Markdown
Contributor

Fixes #2314.

--target_name reaches the WebSocket generator as the name option (garak/cli.py:660-663), but WebSocketGenerator.__init__ only read the uri from its uri argument or option, so it kept the default wss://echo.websocket.org. A run against ws://localhost:3000/chat sent every prompt to the public echo server and scored the echoes. The docs say the uri "can also be passed with --target_name" (docs/source/generators/websocket.rst:13).

Now, when no uri was passed or configured and name is a ws:// or wss:// URL, it is used as the uri. An explicitly configured uri still wins, and with nothing set the echo default is unchanged (test_init_no_uri).

Why this isn't a tiny one-off PR

It changes which server garak talks to, so whether a WebSocket scan tests the target at all. That's a behaviour fix for a documented CLI option, with tests (AGENTS.md:54).

Not a duplicate

gh issue list / gh pr list --repo nvidia/garak --state all for "websocket target_name", "websocket uri", "WebSocketGenerator" and "echo.websocket.org": no issue or PR covers this. #2263 is a different WebSocket bug (empty outputs after the first prompt). #1589 changes cli.py but not the target name handling.

Tests

python -m pytest tests/generators/test_websocket.py tests/generators/test_generators.py
142 passed
python -m pytest tests/generators/
457 passed, 17 skipped

test_init_target_name_sets_uri covers both config shapes the CLI writes (generators.websocket.name and generators.websocket.WebSocketGenerator.name) and fails on main; test_init_configured_uri_wins_over_name pins the explicit uri. black --check is clean.

AI tools helped write this change and the tests. I reviewed every changed line and ran the tests above myself.

@jmartin-tech jmartin-tech left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a good approach though in general this particular experimental generator has significant additional requirement for configuration that may merit a future enhancement to ensure a fully usable specification of the target was provided.

@jmartin-tech jmartin-tech self-assigned this Oct 16, 2026
@jmartin-tech
jmartin-tech merged commit 7852f01 into NVIDIA:main Oct 16, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WebSocket generator ignores --target_name and connects to the echo server

2 participants