feat(ilp): add support for configuring multiple possible database endpoints#6074
feat(ilp): add support for configuring multiple possible database endpoints#6074bluestreak01 merged 77 commits intomasterfrom
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughAdds write-acceptance signaling to configs and settings, introduces HTTP 421 handling, gates ILP writes when not accepting, extends HTTP client API and errno reporting, implements multi-host HTTP line sender with retry/rotation and reset(), and updates tests accordingly. Minor utilities added (IntList factory, Rnd sync), and TCP path guarded. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant App as Application
participant Builder as Sender.Builder
participant LHS as AbstractLineHttpSender
participant HC as HttpClient
participant S1 as Server A
participant S2 as Server B
App->>Builder: address(host1), port(p1), address(host2), port(p2)
Builder-->>App: build() -> HTTP sender (multi-host)
App->>LHS: table(...).symbol(...).flush()
alt No protocol/version yet
LHS->>HC: GET /settings on host[current]
HC-->>LHS: 200 JSON { protocol, accepting.writes }
LHS-->>App: proceed
end
LHS->>HC: POST /imp (lines) to host[current]
alt 204 No Content
HC-->>LHS: 204
LHS-->>App: ok
else 421 Misdirected/Not accepting
HC-->>LHS: 421
LHS->>LHS: rotate to next host
LHS->>HC: retry POST on next host
HC-->>LHS: 204
LHS-->>App: ok
else 5xx retriable
HC-->>LHS: 5xx
LHS-->>App: throw LineSenderException(retryable=true)
end
sequenceDiagram
autonumber
participant Client as HTTP Client
participant Req as Request
participant Net as Network
participant TLS as TLS
Client->>Req: send(host, port, timeout)
Req->>Net: connect(host, port)
alt connect error
Net-->>Req: errno
Req-->>Client: HttpClientException(errno)
else connect ok
Req->>TLS: start if enabled
alt TLS error
TLS-->>Req: errno
Req-->>Client: HttpClientException(errno)
else TLS ok
Req->>Net: write request, read response
Net-->>Req: status/headers/body
Req-->>Client: ResponseHeaders
end
end
sequenceDiagram
autonumber
participant Sender as Line HTTP Sender
participant Server as HTTP Server
participant Proc as LineHttpProcessor
participant Engine as Cairo Engine
Sender->>Server: POST /imp (lines)
Server->>Proc: onHeadersReady()
Proc->>Engine: config.isAcceptingWrites()
alt Not accepting
Engine-->>Proc: false
Proc-->>Server: Status NOT_ACCEPTING_WRITES (421)
Server-->>Sender: 421
else Accepting
Engine-->>Proc: true
Proc-->>Server: proceed to ingest
Server-->>Sender: 204
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🧪 Early access (Sonnet 4.5): enabledWe are currently testing the Sonnet 4.5 model, which is expected to improve code review quality. However, this model may lead to increased noise levels in the review comments. Please disable the early access features if the noise level causes any inconvenience. Note:
Comment |
# Conflicts: # core/src/main/java/io/questdb/cutlass/line/http/AbstractLineHttpSender.java # core/src/main/java/io/questdb/cutlass/line/http/LineHttpSenderV1.java # core/src/main/java/io/questdb/cutlass/line/http/LineHttpSenderV2.java
also - treat all network I/O as equal, no need to differentiate betwen error types. kiss: network error while talking to a server? -> try another one
core/src/main/java/io/questdb/cutlass/line/http/AbstractLineHttpSender.java
Show resolved
Hide resolved
core/src/main/java/io/questdb/cutlass/line/http/AbstractLineHttpSender.java
Outdated
Show resolved
Hide resolved
core/src/main/java/io/questdb/cutlass/line/http/AbstractLineHttpSender.java
Outdated
Show resolved
Hide resolved
core/src/main/java/io/questdb/cutlass/line/http/AbstractLineHttpSender.java
Outdated
Show resolved
Hide resolved
core/src/test/java/io/questdb/test/cutlass/http/line/LineHttpMultiUrlTest.java
Show resolved
Hide resolved
core/src/test/java/io/questdb/test/cutlass/http/line/LineHttpMultiUrlTest.java
Show resolved
Hide resolved
[PR Coverage check]😍 pass : 225 / 250 (90.00%) file detail
|
tandem with https://github.com/questdb/questdb-enterprise/pull/669
This PR adds support to the Java ILP client (and the QuestDB server) to handle multiple possible endpoints. On first connection, the client will poll
/settingsand check if the instance is read-write or read-only. Additionally, if this changes behind the scenes after the sender has been initialised, then the sender will automatically retry HTTP sends on 404 or 421 responses it receives.