Repository navigation
Add Support to mariadbbackup - #273
harunkucuk5 wants to merge 16 commits into
Conversation
mmontes11
left a comment
mmontes11
left a comment
There was a problem hiding this comment.
Hey there @harunkucuk5 ! Huge kudos for starting working on this, looks promising. It will be a massive contribution for the project 🎖️ 💯 ❤️ 💯 ❤️ 🎖️
Note: scheduled backups are not supported for now as the physical backup requires to be triggered on the database pod and when cronjob starts there is nothing that can trigger backup on the database pod. But for the on-demand backups mariadb-backup reconciler triggers the backup on the database pod
Regarding the scheduled backups, I would say don't stress too much, we can take care of it as a separate issue or even implement it myself later. Having mariabackup support is a huge step forward.
I think this PR is attempting to automate this suggestion from @kvaps, is that correct? Kudos to him as well for this fantastic guide 🏅
I am curious about how are you planning to support the physical backup restoration? I think you are in a point where you have the physical backup already in a PVC, how are you planning to ship it to a replica?
- Send the backup through streaming to a replica
- Mounting the
mariabackupPVC
Once the replica has access to the physical backup, how do you plan to actually copy it to /var/lib/mysql:
- By executing a remote command and follow the @kvaps approach?
- By introducing and extra initContainer and restart the
StatefulSet
Having an initial support for mariabackup would be massive, but in the long term we might want to use it to fix replication scenarios, see: #246 (comment)
Left a couple of suggestions in the PR!
Again, thanks a lot for working on this, you have my full support. Let me know if you need a hand for the implementation! 🙏🏻
| if podIP != "" { | ||
| // Start mariadb-backup on the primary pod | ||
| // TODO: make the backup source configurable primary or replicas | ||
| // TODO: add s3 configuration parameters to get backup to s3 directly |
There was a problem hiding this comment.
S3 backup storage is in our roadmap already and something that our community is looking forward to have: #6
If you feel like the PR gets too big feel free to open a new one, or we can take care of it as a separate issue, I leave it up to you. We probably would need to generalize it, not only for physical but also for logical backups.
There was a problem hiding this comment.
well, my initial idea was using the same logic that we have in xtrabackup + xbcloud here that percona has built-in support to get backup directly to s3 and also restore as well. However, it seems there is no such utility like mariadbcloud utility for mariadbbackup. I need to think about this a bit more.
Just as an idea, I could use xtrabackup image in the job pod and it would receive mariabackup stream through socat and upload to bucket with xbcloud like mariadbbackup + xbcloud. As mariadbbackup is ported from xtrabackup hopefully they are compatible.
There was a problem hiding this comment.
Regarding S3, we have a builtin mechanism based of the mariadb-operator image:
So we don't need to rely on xbcloud
Hey @mmontes11, thanks for your great review! Regarding your questions, this will be a backup and restore solution for now, so it won't handle scaling up an existing cluster and copying data from master to replica to fix replication for now, but we can implement this as well later on. So, this PR creates persistent volume if it is not already there and puts backups to that volume. When it comes to restoring I am planning to introduce the init container which will be responsible for the MariaDB backup preparation and copying it to the data folder then it will complete its task. When the actual container starts to run, it will have the data already in the data folder and just start running. |
638530a to
dbe2274
Compare
617f590 to
ece109c
Compare
harunkucuk5
changed the title
|
This PR is stale because it has been open 30 days with no activity. |
|
This PR is stale because it has been open 30 days with no activity. |
|
@harunkucuk5 sorry, may I ask an update about this? mariabackup perhaps the most wanted awesome feature |
|
@pasztorl I encourage you to take a look at this: Following a similar approach as @harunkucuk5, we are intentding to refactor this and support PITR based on mariabackup + binary logs. |
sounds even better :) |
|
@mmontes11 , @pasztorl, sorry that I could not find time to focus on this again. |
|
@harunkucuk5 no prob! I will use and refactor these changes to implement PITR and add you as a co-author if that's ok |
|
This PR is stale because it has been open 60 days with no activity. |
|
This PR is stale because it has been open 60 days with no activity. |
|
This PR is stale because it has been open 60 days with no activity. |
|
This PR was closed because it has been stalled for 30 days with no activity. |
|
This has been supported via this PR: #1266 Thanks for raising the initial changes @harunkucuk5 ! You have been added as co-author. |
Closes #11
This PR adds mariadbbackup support.
Note: scheduled backups are not supported for now as the physical backup requires to be triggered on the database pod and when cronjob starts there is nothing that can trigger backup on the database pod. But for the on-demand backups mariadb-backup reconciler triggers the backup on the database podThe new logic allows scheduled mariadbbackups. Thanks @kvaps.
logs from backup
logs from restore