Skip to content

fix: sanitize NATS JetStream consumer name (replace dots with dashes)#1131

Merged
yokofly merged 2 commits intodevelopfrom
bugfix/fix-nats-consumer-name-dots
Mar 16, 2026
Merged

fix: sanitize NATS JetStream consumer name (replace dots with dashes)#1131
yokofly merged 2 commits intodevelopfrom
bugfix/fix-nats-consumer-name-dots

Conversation

@yokofly
Copy link
Copy Markdown
Collaborator

@yokofly yokofly commented Mar 16, 2026

Summary

NATS consumer names must not contain . (dots) or spaces. This is enforced by the NATS server because dots are the subject token delimiter in NATS subject-based addressing (e.g. orders.us.east). Allowing dots in consumer names would create ambiguity with the subject namespace. The NATS server rejects any js_PullSubscribe call with a consumer name containing . with an Invalid Argument error.

The auto-generated consumer name in Proton uses the format proton-{query_id}. For materialized views, the query ID contains dots (e.g. .mv-95861997-136d-4030-8298-f68745e50f04), producing names like:

proton-.mv-95861997-136d-4030-8298-f68745e50f04
       ^
       dot -- rejected by NATS server

This causes CREATE MATERIALIZED VIEW against a NATS JetStream external stream to fail with:

Code: 2629. DB::Exception: Failed to create JetStream pull subscription:
  subject='orders.>' stream='ORDERS'
  consumer='proton-.mv-95861997-136d-4030-8298-f68745e50f04': Invalid Argument.

Fix: Sanitize the auto-generated consumer name by replacing . and spaces with - before passing it to the NATS C client. This applies to both user-specified and auto-generated consumer names. The sanitized name proton--mv-95861997-136d-4030-8298-f68745e50f04 is valid and works correctly.

Reference: NATS naming conventions -- consumer names follow the same rules as stream names: alphanumeric, -, _, no . or spaces.

Test plan

  • CREATE MATERIALIZED VIEW ... FROM nats_jetstream_stream succeeds without explicit consumer_name
  • MV correctly consumes all messages and routes to target Proton stream
  • Live INSERT via HTTP -> NATS -> MV -> archive stream works end-to-end
  • Explicit consumer_name with dots (e.g. my.consumer) is also sanitized

@yokofly yokofly marked this pull request as ready for review March 16, 2026 03:10
@yokofly yokofly requested a review from yuzifeng1984 March 16, 2026 03:18
/// Generate consumer name: user-specified or auto from query ID
/// Generate consumer name: user-specified or auto from query ID.
/// NATS consumer names must not contain '.' or ' ', so sanitize.
String consumer_name = getConsumerName();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may need initial consumer name in ctor and update getConsumerName() for consistency

@yokofly yokofly force-pushed the bugfix/fix-nats-consumer-name-dots branch from b11ad39 to 72e8ffd Compare March 16, 2026 05:01
@yokofly yokofly force-pushed the bugfix/fix-nats-consumer-name-dots branch from 72e8ffd to b048276 Compare March 16, 2026 05:22
NATS consumer names must not contain '.' or ' '. The auto-generated
consumer name (proton-{query_id}) includes dots from materialized view
query IDs (e.g. .mv-xxxx), causing "Invalid Argument" errors when
creating MV subscriptions. Replace invalid characters with dashes.

Move sanitization to validateSettings() for user-specified names so
getConsumerName() always returns a valid NATS name. Pass the resolved
consumer name explicitly to NATSJetstreamSource for consistency across
stall recovery and fetch subscriptions.

Add NATS JetStream smoke test suite (0032_external_stream_nats) with
coverage for basic read/write, table(), RawBLOB, virtual columns,
materialized views, and drop/recreate. Add NATS + nats-setup services
to the p1k1 Docker Compose deployment.

Co-Authored-By: Claude <[email protected]>
Made-with: Cursor
@yokofly yokofly force-pushed the bugfix/fix-nats-consumer-name-dots branch from b048276 to af0f860 Compare March 16, 2026 05:33
@yokofly
Copy link
Copy Markdown
Collaborator Author

yokofly commented Mar 16, 2026

flaky ?

Some tests failed
Group 3, Suite join_alignment: Failed after 3 retries
Failed cases: 0_append_asof_join_versioned_kv_with_processing_time_alignment

@yokofly yokofly merged commit 9a3959b into develop Mar 16, 2026
8 of 11 checks passed
@yokofly yokofly deleted the bugfix/fix-nats-consumer-name-dots branch March 16, 2026 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants