Skip to content

Try to fix a race in psync2 test#11553

Merged
oranagra merged 1 commit intoredis:unstablefrom
oranagra:try-fix-psync2-test-race
Nov 30, 2022
Merged

Try to fix a race in psync2 test#11553
oranagra merged 1 commit intoredis:unstablefrom
oranagra:try-fix-psync2-test-race

Conversation

@oranagra
Copy link
Member

@oranagra oranagra commented Nov 28, 2022

This test sets the master ping interval to 1 hour, in order to avoid pings in the replicatoin stream incrementing the replication offset, however, it didn't increase the repl-timeout so on slow machines where the test took more than 60 seconds, the replicas would drop and reconnect.

*** [err]: PSYNC2: Partial resync after restart using RDB aux fields in tests/integration/psync2.tcl
Replica didn't partial sync

The test would detect 4 additional partial syncs where it expects only one.

This test sets the master ping interval to 1 hour, in order to avoid
pings in the replicatoin stream incrementing the replication offset,
however, it didn't increase the repl-timeout so on slow machines where
the test took more than 60 seconds, the replicas would drop and
reconnect.

The test named `Partial resync after restart using RDB aux fields` would
detect 4 additional partial syncs where it expects only one.
@oranagra
Copy link
Member Author

observed in https://github.com/redis/redis/actions/runs/3565854385/jobs/5991547180

2022-11-28T14:52:32.0143660Z [err]: PSYNC2: Partial resync after restart using RDB aux fields in tests/integration/psync2.tcl
2022-11-28T14:52:32.0144112Z Replica didn't partial sync

2022-11-28T14:52:31.8558711Z prev sync_full: 0
2022-11-28T14:52:31.8559325Z prev sync_partial_ok: 4
2022-11-28T14:52:31.8559865Z prev sync_partial_err: 0
2022-11-28T14:52:31.8560857Z # Stats
2022-11-28T14:52:31.8565402Z sync_full:0
2022-11-28T14:52:31.8565660Z sync_partial_ok:8
2022-11-28T14:52:31.8565943Z sync_partial_err:0

all replicas report MASTER timeout: no data nor PING received

Copy link
Contributor

@enjoy-binbin enjoy-binbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great, i have seen it a few times, but never delved into it

@oranagra oranagra merged commit b0250b4 into redis:unstable Nov 30, 2022
@oranagra oranagra deleted the try-fix-psync2-test-race branch November 30, 2022 20:03
oranagra added a commit that referenced this pull request Dec 12, 2022
This test sets the master ping interval to 1 hour, in order to avoid
pings in the replicatoin stream incrementing the replication offset,
however, it didn't increase the repl-timeout so on slow machines
where the test took more than 60 seconds, the replicas would drop
and reconnect.

```
*** [err]: PSYNC2: Partial resync after restart using RDB aux fields in tests/integration/psync2.tcl
Replica didn't partial sync
```

The test would detect 4 additional partial syncs where it expects
only one.

(cherry picked from commit b0250b4)
oranagra added a commit that referenced this pull request Dec 12, 2022
This test sets the master ping interval to 1 hour, in order to avoid
pings in the replicatoin stream incrementing the replication offset,
however, it didn't increase the repl-timeout so on slow machines
where the test took more than 60 seconds, the replicas would drop
and reconnect.

```
*** [err]: PSYNC2: Partial resync after restart using RDB aux fields in tests/integration/psync2.tcl
Replica didn't partial sync
```

The test would detect 4 additional partial syncs where it expects
only one.

(cherry picked from commit b0250b4)
madolson pushed a commit to madolson/redis that referenced this pull request Apr 19, 2023
This test sets the master ping interval to 1 hour, in order to avoid
pings in the replicatoin stream incrementing the replication offset,
however, it didn't increase the repl-timeout so on slow machines
where the test took more than 60 seconds, the replicas would drop
and reconnect.

```
*** [err]: PSYNC2: Partial resync after restart using RDB aux fields in tests/integration/psync2.tcl
Replica didn't partial sync
```

The test would detect 4 additional partial syncs where it expects
only one.
enjoy-binbin pushed a commit to enjoy-binbin/redis that referenced this pull request Jul 31, 2023
This test sets the master ping interval to 1 hour, in order to avoid
pings in the replicatoin stream incrementing the replication offset,
however, it didn't increase the repl-timeout so on slow machines
where the test took more than 60 seconds, the replicas would drop
and reconnect.

```
*** [err]: PSYNC2: Partial resync after restart using RDB aux fields in tests/integration/psync2.tcl
Replica didn't partial sync
```

The test would detect 4 additional partial syncs where it expects
only one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants