RReliableTopic subscriber stops consuming permanently after one failed watchdog renewal or status update (no retry; then "ERR Invalid stream ID") #7391

Summary:

If Redis/Valkey rejects writes for a short time (for example while a replica reconnects), an RReliableTopic listener can stop receiving messages forever. Redisson tries once to refresh the subscriber's registration or status, and if that one attempt fails it never tries again. The listener then quietly stops reading. No error reaches the application, and messages pile up unread until the service is restarted.

Expected behavior

A transient write failure, such as NOREPLICAS while a replica reconnects, should not permanently stop an RReliableTopic listener. Once the server accepts writes again, the subscriber should keep its registration (or re-create it) and keep consuming messages.

Actual behavior

There are two related failure paths in RedissonReliableTopic. Both stop the poll loop permanently. No exception reaches the caller, and the subscriber still looks registered until it dies.

1. A failed watchdog renewal is never rescheduled.
renewExpiration() reschedules itself only when the renewal succeeds. When the renewal script fails, for example with NOREPLICAS, it logs this once and never renews again:

ERROR org.redisson.RedissonReliableTopic - Can't update reliable topic testFailedRenewal expiration time
org.redisson.client.RedisNoReplicasException: NOREPLICAS Not enough good replicas to write. script: 422be3f551b99935463b2c32454148831f31afd9, on @user_script:1..

The subscriber's score in {<name>}:timeout stops moving and expires after reliableTopicWatchdogTimeout. Nothing is visible yet: the blocked XREADGROUP still delivers.

On the next message, the listener receives it. Then the post-poll status-update script runs zrangebyscore … 0 now-1, finds the subscriber's own expired id, and destroys its own consumer group. With no groups left, score stays 92233720368547758. Lua 5.1 tostring() formats that as 9.2233720368548e+16, so the script calls XRANGE <stream> 9.2233720368548e+16-0 +, and that fails:

ERROR org.redisson.RedissonReliableTopic - Unable to update subscriber status
org.redisson.client.RedisException: ERR Invalid stream ID specified as stream command argument script: e584bfe0230f6e59051ef5a8a125585d0f7f1c94, on @user_script:1..

The updateFuture error branch returns without calling poll(id), so the subscriber is dead. Later messages stay unread in the stream. The topic now has no consumer group and no {name}:timeout entry, but listeners still contains the listener, and subscribed is still true, so addListener on the same instance will not re-subscribe.

(You can confirm the formatting with EVAL "return tostring(92233720368547758) .. '-0'" 0, which returns 9.2233720368548e+16-0.)

2. A failed status update stops polling even if renewals keep working.
This needs no renewal failure. Any single failure of the post-poll update script stops polling, because poll(id) is not rescheduled after "Unable to update subscriber status". In the test below it is NOPERM for a few ms:

ERROR org.redisson.RedissonReliableTopic - Unable to update subscriber status
org.redisson.client.RedisException: NOPERM User default has no permissions to run the 'evalsha' command.

Renewals keep succeeding, so the consumer group and the {name}:timeout entry stay valid and unexpired. The subscriber looks healthy, but it never reads again: last-delivered-id stays put while the stream grows.

These look like modes 5 and 6 in the contributor analysis on #7262 (Aug 22), which were listed as not covered by #7263, #7310 or #7300. #7299 / #7300 (4.8.0) fixed only the listener-exception path. Both tests below still fail on 4.8.0.

Steps to reproduce or test case

See attachment

reproduce.zip

Redis version

Production: Valkey 8.1.10, cluster mode, TLS, 1 primary + 1 replica, min-replicas-to-write 1, min-replicas-max-lag 10. Reproduced with the tests above on redis:8.10 and valkey/valkey:8.1.10-alpine3.24, standalone.

Redisson version

Reproduced on 4.8.0.

Redisson configuration

Production: ClusterServersConfig, readMode = MASTER, TLS (mTLS with short-lived client certificates), default reliableTopicWatchdogTimeout (600 000 ms). Tests: single server, default config, reliableTopicWatchdogTimeout = 3000 for the renewal test.

Production trigger: a replica lost its replication link for about 4 minutes, so writes were rejected with NOREPLICAS. Every reliable-topic subscriber across several services logged "Can't update reliable topic … expiration time" in that window. Then each one died on its first message after expiry, up to 23 hours later. Only a restart recovered them.

reproduce.zip