Skip to content

Commit 7df1b64

Browse files
committed
fix(opensearch-operator): bound the calls that take a context from the caller
The first pass bounded the 15 calls the client makes on its own behalf and left the 28 that take a context from the caller to the transport. That is not a bound on them: ResponseHeaderTimeout stops counting once the headers arrive, so a body that starts and then stops waits for as long as the process lives, and nothing on the reconcile path puts a deadline on the context it hands in. One of those 28 is readVotingConfigState, reached from the bootstrap-retire path where the reported silence starts. The five doHTTP helpers now derive the same requestTimeout. The deadline has to outlive the call, because these hand the body back still open, so the cancel rides on the body and is released when the body is closed, which is how http.Client applies its own Timeout. Two tests come with that: a body that stops halfway is abandoned, and a body is still readable after the call that fetched it has returned. Each was checked against the mutation it exists to catch, dropping the deadline and cancelling on return. The stalled-server test kept its accepted connections in a local that went out of scope. net.netFD carries a finalizer that closes the descriptor, so a collection turned the server from one that says nothing into one that resets the connection, and the test passed with ResponseHeaderTimeout removed. The connections are now held for the test's life, and it forces a collection every 10ms to keep that honest: with the bound removed it now fails after the grace period. The Dockerfile entry gains the drop condition every other entry carries, and names the one wait that is left, the product check inside opensearch-go. Assisted-by: LLM Signed-off-by: IvanHunters <[email protected]>
1 parent c5e9062 commit 7df1b64

2 files changed

Lines changed: 334 additions & 50 deletions

File tree

‎packages/system/opensearch-operator/images/opensearch-operator/Dockerfile‎

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -28,13 +28,18 @@
2828
# patches/timeout-cluster-api-calls.diff bounds the calls the operator makes to
2929
# a cluster's HTTP API. Upstream builds its own http.Transport, which carries no
3030
# dial, handshake or response-header timeout; it passes context.Background() to
31-
# every call it makes on its own behalf; and it configures no http.Client, so
32-
# there is no timeout there either. A cluster that accepts a connection and then
33-
# says nothing therefore holds the call for as long as the process lives, and
34-
# with one worker per controller that one call stops every OpenSearchCluster
35-
# from being reconciled. The deletion that then waits out its whole termination
36-
# timeout is the visible end of it. Tracked as cozystack/cozystack#4337; it
37-
# belongs upstream, and there is no upstream issue for it yet.
31+
# every call it makes on its own behalf; the calls that take a context from the
32+
# caller are handed one with no deadline on it; and no http.Client is
33+
# configured, so there is no timeout there either. A cluster that accepts a
34+
# connection and then says nothing therefore holds the call for as long as the
35+
# process lives, and with one worker per controller that one call stops every
36+
# OpenSearchCluster from being reconciled. The deletion that then waits out its
37+
# whole termination timeout is the visible end of it. One wait is left, inside
38+
# opensearch-go: the product check on a client's first request carries a
39+
# context.Background() the call site cannot reach, and the transport's bounds
40+
# cover it as far as the response headers. Tracked as cozystack/cozystack#4337;
41+
# it belongs upstream, there is no upstream issue for it yet, and the patch goes
42+
# away with the first release that carries a fix.
3843
#
3944
# The patches are applied in the order the glob expands, which is alphabetical.
4045
# skip-unusable-drain-exclusion builds on restart-on-permanently-yellow, and

0 commit comments

Comments
 (0)