Skip to content

Conversation

@jschaul
Copy link
Member

@jschaul jschaul commented Apr 21, 2022

This PR:

  • allows to configure gundeck to write to an additional redis (cluster or master mode); optionally. Reads will happen from the main redis.
  • adds a test for the redis-migration scenario: configure to write to two redises (old and new); insert some data; configure only the new redis as default redis, read, ensure data is present.
  • adds redis-cluster in docker-compose for local integration tests (a little more CPU/RAM needed)
  • adds redis-cluster wrapper helm chart for CI/kubernetes-based integration tests (and more generally also for on-premise deployments). redis-cluster requires dynamic persistent volume provisioning; which was introduced in our kube-ci cluster with https://github.com/zinfra/cailleach/pull/1037

Drive-by refactoring change (not strictly needed on this PR):

Related to https://github.com/zinfra/cailleach/pull/1021

This is needed to perform a smooth no-downtime migration from one redis to another as detailed in https://github.com/zinfra/cailleach/blob/master/targets/wire/notes-redis-migration.md#how-to-migrate-traffic-from-redis-in-ec2classic-to-redis-in-eks

Checklist

  • The PR Title explains the impact of the change.
  • The PR description provides context as to why the change should occur and what the code contributes to that effect. This could also be a link to a JIRA ticket or a Github issue, if there is one.
  • changelog.d contains the following bits of information (details):
    • A file with the changelog entry in one or more suitable sub-sections. The sub-sections are marked by directories inside changelog.d.
    • If new config options introduced: added usage description under docs/reference/config-options.md

akshaymankar and others added 5 commits April 21, 2022 21:35
establish connection and thread it into Gundeck's Env
I, starting connection to main-redis..., connectionMode=Cluster, connInfo=ConnInfo {connectHost = "127.0.0.1", connectPort = PortNumber 6377, connectAuth = Nothing, connectDatabase = 0, connectMaxConnections = 100, connectMaxIdleTime = 30s, connectTimeout = Just 5s, connectTLSParams = Nothing}
I, starting connection to redis in cluster mode ...
gundeck: ConnectionLost
@jschaul jschaul requested a review from akshaymankar April 21, 2022 20:23
@jschaul jschaul temporarily deployed to cachix April 21, 2022 20:23 Inactive
@akshaymankar akshaymankar temporarily deployed to cachix April 25, 2022 10:35 Inactive
@akshaymankar akshaymankar temporarily deployed to cachix April 25, 2022 15:02 Inactive
@jschaul jschaul temporarily deployed to cachix April 25, 2022 22:49 Inactive
@akshaymankar akshaymankar temporarily deployed to cachix April 26, 2022 10:30 Inactive
* volumePermissions.enable make it run an initContainer which fixes the
permissions on the mounted volume, so it can be accessed by the non-root user
which runs redis.

* persistence.size is set to 100 Mi, so it doesn't kill the cluster.
@akshaymankar akshaymankar temporarily deployed to cachix April 26, 2022 12:59 Inactive
@akshaymankar akshaymankar temporarily deployed to cachix April 26, 2022 15:11 Inactive
@jschaul jschaul temporarily deployed to cachix April 26, 2022 16:35 Inactive
@jschaul jschaul marked this pull request as ready for review April 26, 2022 16:39
@jschaul jschaul temporarily deployed to cachix April 26, 2022 16:39 Inactive
@jschaul jschaul merged commit 6e0dd20 into develop Apr 27, 2022
@jschaul jschaul deleted the additional-redis-write branch April 27, 2022 11:08
@smatting smatting mentioned this pull request Nov 7, 2025
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants